ADVANCED EYESIGHT DEVICE FOR VISUALLY IMPAIRED

Abstract

An eyesight device for the visually impaired comprises a case having a front, rear and side walls. An ultrasonic sensor is provided on at least the front wall although, preferably, ultrasonic sensors are provided on the front and side walls to provide a greater range of peripheral vision. A microcontroller within the case is coupled to the ultrasonic sensor. A power source energizes the microcontroller. An audible signal generating means is adapted to generate an audible signal. The microcontroller is programmed to receive the output of the ultrasonic sensor and computing the distance between the case and an obstacle spaced from the case and energize the audible signal generating means when the distance is less than a preselected threshold distance. Attaching means is provided for attaching the case to an item of clothing of the user to free the hands of the user.

Claims

1. An eyesight device for the visually impaired comprises a case having a front, rear and side walls; an ultrasonic sensor on at least said front wall; a microcontroller within said case coupled to said ultrasonic sensor; a power source for energizing said microcontroller; and audible signal generating means for generating an audible signal, said microcontroller being programmed to receive the output of said ultrasonic sensor and compute the distance between said case and an obstacle spaced from said case and energize said audible signal generating means when said distance is less than a preselected threshold distance; and attaching means for attaching said case to an item of clothing of the user.

2. A device as defined in claim 1, wherein said microcontroller comprises an Arduino controller.

3. A device as defined in claim 1, wherein said attaching means comprises a spring-loaded clip.

4. A device as defined in claim 3, wherein said ultrasonic sensor is mounted on said front wall and said clip is mounted on said rear wall.

5. A device as defined in claim 1, wherein said ultrasonic sensor is mounted on said front wall and further ultrasonic sensors are mounted on said side walls.

6. A device as defined in claim 1, wherein said microcontroller is programmed to set a predetermined threshold distance and to energize said audible signal generating means only when said ultrasonic sensor measures a distance to an obstacle less than said threshold distance.

7. A device as defined in claim 6, wherein said microcontroller is programmed to issue an audible signal having a pitch that is a function of the distance of an obstacle from said ultrasonic sensor.

8. A device as defined in claim 1, wherein said microcontroller is programmed to mate or sync with a smartphone app to transmit distance data to the smartphone to generate alerts on the smartphone.

9. A device as defined in claim 1, further comprising vibrating means connected to said microcontroller to vibrate said case when a distance less than said threshold distance is detected.

10. A device as defined in claim 1, wherein said vibrating means comprises a vibrating motor.

11. A device as defined in claim 1, further comprising a source of light connected to said microcontroller to generate visible light when a distance less than said threshold distance is detected.

12. A device as defined in claim 11, wherein said source of light comprises at least one LED.

13. A device as defined in claim 1, wherein said audible signal generating means comprises a Piezo element or buzzer.

14. A device as defined in claim 1, further comprising an image sensor connected to said microcontroller for generating an image of an obstacle detected by said ultrasonic sensor.

15. A device as defined in claim 15, wherein said image sensor comprises a camera.

16. A device as defined in claim 15, in combination with a smartphone coupled or mated with the device, and connectivity means for transmitting distance data calculated by said microcontroller to said smartphone.

17. A device as defined in claim 16, wherein said smartphone is provided with an app that translates or converts distance data to verbal alerts.

18. A device as defined in claim 16, wherein said smartphone is programmed to recognize at least one of shape, size, direction of movement of an obstacle detected by said ultrasonic sensor and to issue verbal alerts identifying information characterizing the obstacle.

19. A device as defined in claim 16, wherein said connectivity means comprises Bluetooth connectivity.

20. A device as defined in claim 1, wherein said microcontroller is programmed to enable setup of distance threshold prior to use of the device.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The above and other aspects, features and advantages of the present invention will be more apparent from the following description when taken in conjunction with the accompanying drawings, in which:

[0025] FIG. 1 is a diagrammatic perspective view of a device for assisting the visually impaired people in accordance with the invention;

[0026] FIG. 2 is an exploded view of the device shown in FIG. 1, showing the major components incorporated into the device;

[0027] FIG. 3 is similar to FIG. 1, showing an alternate embodiment that incorporates an image sensor and speaker;

[0028] FIG. 4 is similar to FIGS. 1 and 3, showing a still further alternate embodiment that incorporates multiple sensors;

[0029] FIG. 5 is a block circuit diagram corresponding to the device shown in FIG. 1;

[0030] FIG. 6 is an illustration of the bread boarded components corresponding to the blog diagram shown in FIG. 5;

[0031] FIG. 7 is a flowchart illustrating the operation of the embodiment represented in FIGS. 5 and 6;

[0032] FIG. 8 is similar to FIG. 5 for an alternate embodiment of the device;

[0033] FIG. 9 is similar to FIG. 6 for the embodiment shown in FIG. 8;

[0034] FIG. 10 is similar to FIG. 7 for the embodiment shown in FIGS. 8 and 9;

[0035] FIG. 11 is a perspective image of the embodiment illustrated in FIGS. 5 and 6;

[0036] FIG. 12 is similar to FIGS. 5 and 8 for a still further embodiment of the device;

[0037] FIG. 13 is similar to FIGS. 6 and 9 for the embodiment shown in FIG. 12; and

[0038] FIG. 14 is similar to FIGS. 7 and 10 for the embodiment shown in FIG. 12.

DETAILED DESCRIPTION

[0039] Referring now specifically to the Figures, in which identical or similar parts are designated by the same reference numerals throughout, and first referring to FIGS. 1 and 2, a device for assisting the visually impaired in accordance with the invention is generally designated by the designation D.

[0040] The device D includes a microcontroller 10, and ultrasonic sensor 20 and a Bluetooth controller 30. The microcontroller 10 uses an Arduino circuit that may be mounted on a printed circuit board (PCB or breadboard). The ultrasonic sensor 20 detects obstacles and measures distances by sending out ultrasonic signals and receiving and detecting signals reflected from items or obstacles. In FIG. 1 multiple ultrasonic sensors 20, 20′ or shown for performing such functions in multiple directions to detect obstacles not only in front of the user or the user's direction of movement but also to the sides of the user. Although ultrasonic sensors are only shown on can surfaces of the device D it will be clear that additional ultrasonic sensors may be used and placed on additional surfaces so that alerts may be received not only to address obstacles in front of the user but also to both sides of the user.

[0041] The device includes a PSO electric capsule or PSO buzzer 40 issue audible alerts. The aforementioned components form a circuit 50 that can be integrated in a PCB board mounted in a case 60. In FIG. 1, the case 60 is shown as a substantially rectangular housing having a front wall 60 a, sidewalls 60 B, 60 C, a top wall 60 D and a rear wall 60 E. The ultrasonic sensor 20 is mounted on the front wall 60 a. As indicated above, additional ultrasonic sensors 20′ can be mounted on the sidewalls 60 B and 60 C. While shown as a substantially rectangular housing this is not critical and the case 60 may take on different configurations.

[0042] The circuit 50 is powered by a battery 70. Any battery module may be used, including rechargeable batteries that can be recharged by using a conventional AC converter or adapter. To avoid excessive or unnecessary drain on the battery there is preferably provided an on-off switch 80 so that the device D can be de-energized when not in use.

[0043] Referring to FIGS. 3 and 4 a similar but different embodiment is depicted that includes an image sensor 90 and a speaker 100.

[0044] Referring to FIGS. 5 and 6 the embodiment shown in FIGS. 1 and 2 is illustrated with additional detail. The Arduino microcontroller 10 mounted on the PCB is readily and quickly connected to the ultrasonic sensor 20, the buzzer 40 and the Bluetooth controller 30, as well as to the power source or battery 70. All of these components are housed in the case 60, that also houses the battery 70. The case is provided with a clip 110 on the rear or back surface or wall 60 E so that the device D can be attached and secured to a user's belt or clip to an item of clothing. The specific clip used is not critical and any suitable or known clip may be used depending on the manner in which the device is to be secured to the user. The device can be worn on the belt, the pencil skirt or any item of clothing. Once the device is turned on it will start sensing obstacles in close proximity to the person wearing the device. The person gets closer to an obstacle, or the optical such as another person or vehicle approaches the person, the device will provide an audible alarm. The microcontroller 10 is preferably programmed so that the alert or alarm pitch changes as the obstacle comes closer to the person to provide an indication of proximity. The higher the pitch the close of the obstacle. This feedback can be helpful to the user during navigation.

[0045] Referring to FIG. 7, the flowchart describes the operation of the device D. At the start 200 input and output pins are set up at 202, and the device can be set up to establish a distance a threshold at 204 at which the device will initiate alerts or alarms. The ultrasonic sensor is initialized the 206 after which the ultrasonic sensor sends and receives reflected pulses at 208. The output of the ultrasonic sensor is used to measure the distance to at 210, such distance is then compared to the threshold distance at 212. In the measured distance is at or closer to this threshold distance selected at 204 a signal is generated at 214 to generate beeps or buzzing sounds by the buzzer 40. When used with a smart phone application the device D sends data via the Bluetooth sensor at 216 to the smart phone. At this point, the circuit 50 forces for a short duration at 218 before resuming sending and receiving pulses at 208. When the comparison of the measured distance is compared to the threshold distance at 212 is further or greater than the threshold distance the circuit pauses for a short duration at 218 before recycling and taking additional measurements at 208.

[0046] The present invention contemplates the use of the device D with a smart phone application to provide additional functionalities. Still referring to FIG. 7, when a smart phone app is used the app is started at 222 initialized variables that 222. The user then initializes Bluetooth at 224, after which the app waits for Bluetooth input at 226. When signals are sent to generate beeps or buzzing sounds at 214 that data is sent by the Bluetooth sensor at 216 to be received by the smart phone app at 228. Receipt of such a signal generates an alert on the smart phone at 230. The user, therefore, is provided with two sources of alerts. These can take the form of similar or different sounds, as may be preferred by the user.

[0047] The basic wearable sensor for visually impaired people warns them of such obstacles with sound and haptics It is a lightweight wearable device that can be worn on the body for extended periods of time and frees up the hands of the person to go about their normal activities. The device can be used by people that are fully visually impaired as well as moderately to severely visually impaired. Using off-the shelf components, it is an affordable device while providing significant benefits. It detects objects that are above hip-level or fast-moving and can detect obstacles up to 12 ft. The device can be pre-configured for a distance of 3 feet as the threshold. The device provides audible alerts as obstacles move closer and preferably change the frequency or pitch of the alerts as the distances change providing higher pitch alerts as the obstacles get closer. Being lightweight, the device can be worn for extended periods of time. Importantly, the device can be worn on the belt or dress or other item of clothing thus freeing up the hands of the user without compromising the range of vision of the device thereby maintaining the desired direction(s) of detection irrespective of positions of the user's hands. Also, importantly, the device does not require any supplementary device or support subscription services. Although not critical, the device is preferably adapted to be used with an optional companion smartphone app that communicates via Bluetooth with the device and provides haptic alerts on the phone. Thus, such embodiment:

[0048] 1. Uses an Arduino Uno board along with a full size breadboard;

[0049] 2. Connects an ultrasonic sensor for detecting obstacles and measuring the distance;

[0050] 3. Uses a piezoelectric sensor to generate beeps when the distance is within the predetermined or preset threshold value;

[0051] 4. The board is powered by a 9V battery attached to one side of the board;

[0052] 5. The Bluetooth sensor outputs the distance data using a serial data delivery;

[0053] 6. The smartphone companion app detects the Bluetooth signal and outputs the data on the phone. The app also generates a vibrating alert whenever the distance is within the threshold.

[0054] Referring to FIGS. 8 and 9, modified embodiment is illustrated that, in most respects, is similar to the one described in connection with FIGS. 5 and 6. However, in this modified embodiment the microcontroller 10 is also connected to a light source, such as LED 112 and a vibrating motor 114. Referring to FIG. 10, the flowchart for operating this modified embodiment shown. In addition to the steps previously described in connection with FIG. 7, the vibration motor and LED light are set up at 232 and 234. When a signal is generated to create an alarm or alert the signal is also used to activate or energize the vibration motor and LED at 236. The LED may be useful for those users that are only partially visually impaired but may have the ability to detect changes in brightness. The vibration motor provides a physical feedback that are readily detected and understood by the user.

[0055] Still referring to FIG. 7, the smart phone app may also be programmed to detect and synchronize with only a single user's device. User clicks can be used to reconnect or reinitiate smart phone to a single device at 238, the app reinitializing and clearing previously entered data at 240, making it possible to reset the device ID and reconnect at 242, connecting to the user's specific device at 246. The diagnostic messages can be saved at 248. Now, when data is sent via Bluetooth at 216 and alert will be generated at 230 only when the data is received from the user's specific device.

[0056] The second version of the device and the companion app provide improvements over the first version. The device and the app provide all the features of the first version and uses an Arduino Uno microcontroller directly on a breadboard with minimal circuitry thereby reducing the device size to ½ the original size. The circuitry is also set up such that it does not need a lot of space inside the case. As a result, the device is substantially smaller and even more lightweight so can be worn for even more extended periods. The device uses a small form factor power source with coin-type batteries that are embedded within the casing that reduces the overall size. The device has a hardware on/off switch and uses a piezoelectric sensor to generate beeps when the distance is within the threshold. The pitch also increases or decreases as the object moves closer or farther from the wearer to provide additional input to the user. When the distance of an obstacle is within the threshold, the device also uses a vibration motor to provide haptic alerts using a sub-circuit that includes a transistor for switching. The control program also sends a signal to an LED that lights up when the obstacle is within the threshold distance. The device uses a Bluetooth Low Energy sensor module that outputs the distance data using serial data delivery. The app is now able to connect and disconnect with the device from within the app without restarting the app. It also provides more details for the alerts and provides diagnostic messages for troubleshooting.

[0057] The features of the second embodiment are:

[0058] 1. Uses an Arduino Uno microcontroller directly on a breadboard with minimal circuitry thereby reducing the device size to ½ the original size;

[0059] 2. Uses a small form factor power source that can be embedded within the casing to reduce overall size. The device also has a hardware on/off switch;

[0060] 3. Uses a piezoelectric sensor to generate beeps when the distance is within the preset threshold value;

[0061] 4. When the distance of an obstacle is within the threshold value, the control program also sends a signal to the vibration motor using a sub-circuit that includes a transistor for switching;

[0062] 5. The control program also sends a signal to an LED that lights up when the obstacle is within the threshold distance;

[0063] 6. Uses a Bluetooth Low Energy sensor module that outputs the distance data using serial data delivery;

[0064] 7. The smartphone companion app detects the Bluetooth signal and outputs the data on the phone. The app also generates a vibrating alert whenever the distance is within the threshold distance; and

[0065] 8. The app is now able to connect and disconnect with the device from within the app without restarting the app. It also provides more details for the alerts and provides diagnostic messages for troubleshooting.

[0066] Referring to FIGS. 12 and 13 a further embodiment is shown that is modified to include a camera 116 also connected to the microcontroller 10. In this embodiment, the image sensor or camera 90, used with or without the ultrasonic sensor, is used to detect visual images and interpret the nature of the obstacle using hardware and software to provide additional audible feedback to the user through the speaker 100. When used together with the ultrasonic sensor, the device can provide more accurate feedback to the user. Referring to FIG. 14, many of the steps described in connection with FIGS. 7 and 10 are the same. However, in FIG. 14 the device ID is sent and received at 250, the smart phone app configuring the connection to the specific device at 252 by means of the two-way synchronization between the device and the app at 254. The user can now with threshold distance on the smart phone at 256 and such user selected threshold distance is saved in the device at 258. This configuration is saved on a smart phone at 260, and the device setup the distance and connect properties at 262. To use the camera, the camera module is initialized at 263 and images are captured at 264. Image data is saved at 266 and further images are again captured at 264 in a loop to continuously monitor the images of the approaching obstacle. The saved image properties are sent to the smart phone at 268 and the plot of data is received at 228 the smart phone app associates the data to the image information at 270 in the app generates an alert, calling out the image and/or causing the motor to vibrate at 272. In this embodiment, therefore, the app provides verbal alerts, by means of a conventional voice synthesizer, providing the user with voice descriptions of the nature, size and/or shape of the obstacle detected by the camera. Therefore, the camera 90, connected to the microcontroller 10 can detect visual images and interpret the nature of the obstacle using hardware and software to provide additional audible feedback to the user through a speaker 100. As suggested, the image sensor or camera can be used with or without the ultrasonic sensor to provide more accurate and complete feedback to the user. As also suggested, multiple ultrasonic sensors and/or multiple image sensors mounted at various angles aimed at different directions in relation to the direction of movement of the user can be used to provide a greater area of coverage.

[0067] The third version of the device and the companion app provide all the features of the second version and, in addition, the device can now attempt to use a camera module with image detection and recognition ability to provide computer vision ability. The signals are sent to the companion app for voice callouts via a voice synthesizer. The device and app are now able to synchronize both ways. The companion app can be used to configure the device to set and save settings as well as provide connectivity to a particular device providing device affinity. The device can also save the settings internally to communicate only to that particular smartphone app. The distance threshold can be configured using the app. Once the device and app are linked the app will only try to connect to that mated device until changed. This helps the app on the phone to only connect to a specific device and not to other similar devices nearby. The app can also be used to reset and clear settings to connect to another device. Thus, the circuit and device are now improved for better resiliency and robustness.

[0068] The features of the third embodiment are:

[0069] 1. Uses an Arduino Uno microcontroller directly on a breadboard with minimal circuitry thereby reducing the device size to ½ the original size;

[0070] 2. Uses a small form factor power source that can be embedded within the casing to reduce overall size;

[0071] 3. Uses a piezoelectric sensor to generate beeps when the distance is within the threshold value;

[0072] 4. When the distance of an obstacle is within a preset threshold value, the control program also sends a signal to the vibration motor using a sub-circuit that includes a transistor for switching. The circuit is now improved for better resiliency;

[0073] 5. The control program also sends a signal to an LED that lights up when the obstacle is within the threshold distance;

[0074] 6. The device now attempts to use a camera module with image detection and recognition ability to provide computer vision ability. The signals are sent to the companion app for voice callouts via synthesizer;

[0075] 7. Uses a Bluetooth Low Energy sensor module that outputs the distance data using serial data delivery;

[0076] 8. The smartphone companion app detects the Bluetooth signal and outputs the data on the phone. The app also generates a vibrating alert whenever the distance is within the threshold distance value;

[0077] 9. The app is now able to connect and disconnect with the device from within the app without restarting the app. It also provides more details for the alerts and provides diagnostic messages for troubleshooting;

[0078] 10. The device can now communicate both ways with the companion app. The companion app can be used to configure the device to set and save settings as well as provide connectivity to a particular or specific device. The device will also save the settings internally to communicate only to that particular app; and

[0079] 11. Once the device and app are linked they app will only try to connect to that device until changed. This helps the app on the phone to only connect to my device and not to other devices nearby. The app can also be used to reset and clear settings to connect to another nearby device.

[0080] A hip or body mounted sensor is better suited because of its ability to sense a wider angle with multiple ultrasonic sensors that are pointed in 2, 3 or more different directions, and recognize low lying as well as above hip level objects while freeing up a user's hands to provide a more efficient and safer experience to the user.

[0081] The foregoing is considered as illustrative only of the principles of the invention. Further, since numerous modifications and changes will readily occur to those skilled in the art, it is not desired to limit the invention to the exact construction and operation shown and described, and accordingly, all suitable modifications and equivalents may be resorted to, falling within the scope of the invention.