ASSISTED DRIVING SYSTEM FOR CLEARANCE GUIDANCE
20200023834 ยท 2020-01-23
Inventors
Cpc classification
B60W50/14
PERFORMING OPERATIONS; TRANSPORTING
B60W30/0953
PERFORMING OPERATIONS; TRANSPORTING
G06V20/58
PHYSICS
G05D1/0088
PHYSICS
B60W30/09
PERFORMING OPERATIONS; TRANSPORTING
B60W2554/00
PERFORMING OPERATIONS; TRANSPORTING
B60W30/095
PERFORMING OPERATIONS; TRANSPORTING
G05D1/0276
PHYSICS
International classification
B60W30/09
PERFORMING OPERATIONS; TRANSPORTING
B60W50/14
PERFORMING OPERATIONS; TRANSPORTING
G05D1/00
PHYSICS
B60Q9/00
PERFORMING OPERATIONS; TRANSPORTING
Abstract
An assisted driving system can provide forward clearance guidance for a vehicle by using data about an object attached to the vehicle and data about a space in an expected path of travel of the vehicle. The data about the object can be input into the vehicle's data processing systems through a mobile application on a driver's smartphone, and the mobile application can be configured to exchange information with the vehicle's data processing systems through a wired (e.g. USB) or wireless (e.g., Bluetooth) interface.
Claims
1. A machine implemented method comprising: receiving object data representing an object size of an object placed on a vehicle; storing the object data for use by a data processing system coupled to the vehicle; capturing a first set of data using a set of one or more sensors on the vehicle, the set of one or more sensors coupled to the data processing system to provide data from which the data processing system is configured to determine a size of a space in an expected path of travel of the vehicle; comparing, by the data processing system, the size of the space to the object size to determine whether the vehicle, with the object placed on the vehicle, can pass through the space.
2. The method as in claim 1, further comprising: generating, through a user interface, a warning to the vehicle's user that the vehicle will not pass through the space in response to determining, from the comparing, that the vehicle will not pass through the space.
3. The method as in claim 2, wherein the first set of data comprises one or more images and the set of sensors includes one or more cameras, and wherein the method further comprises: causing the vehicle to stop or change its path if the vehicle continues on the path toward the space after the warning is generated.
4. The method as in claim 3, wherein the method further comprises: generating, from a LiDAR system or other sensors on the vehicle and coupled to the data processing system, ranging data from which the data processing system is configured to determine the distance to the space from both the images and the ranging data.
5. The method as in claim 4, wherein the object data is received through a mobile application on the vehicle user's smart phone and the mobile application is configured to provide the object data to the data processing system through either a wired connection or a wireless connection.
6. The method as in claim 5, wherein the vehicle's user enters the object data through a user interface on the mobile application.
7. The method as in claim 5, wherein the object data is a captured image of the object on the vehicle, the captured image obtained through a camera on the vehicle user's smart phone.
8. The method as in claim 5, wherein the vehicle user's smart phone is configured to unlock the vehicle and allow the vehicle to be operated.
9. The method as in claim 4 wherein the other sensors that generate ranging data include one of radar sensors or ultrasound sensors, and wherein the data processing system is coupled to one or more braking systems to stop or slow the vehicle and is coupled to a steering system to steer the vehicle.
10. An assisted driving system comprising: a vehicle having a set of one or more sensors to capture a first set of data that show an expected path of travel of the vehicle; an interface configured to receive object data from a mobile device, the object data representing an object size of an object placed on the vehicle; a data processing system coupled to the interface to receive and store the object data and coupled to the set of one or more sensors, the data processing system configured to determine a size of a space in the expected path and to determine whether the vehicle, with the object placed on the vehicle, can fit through the space.
11. The assisted driving system as in claim 10 wherein the interface comprises one of a wired connection or a wireless connection.
12. The assisted driving system as in claim 11, wherein the data processing system is coupled to a display device and wherein the display device displays a warning generated by the data processing system in response to determining that the vehicle will not fit through the space.
13. The assisted driving system as in claim 12 wherein the data processing system is configured to cause the vehicle to stop or change its path if the vehicle continues on the path toward the space after the warning is generated.
14. The assisted driving system as in claim 13, further comprising a LiDAR system or other sensors coupled to the data processing system, wherein the data processing system is configured to use ranging data from the LiDAR system or other sensors to determine a distance to the space.
15. The assisted driving system as in claim 14 wherein the object data is received through a mobile application on the vehicle user's smart phone and the mobile application is configured to provide the object data to the data processing system through the interface.
16. The assisted driving system as in claim 15 wherein the user enters the object data through a user interface on the mobile application.
17. The assisted driving system as in claim 15 wherein the object data is a captured image of the object on the vehicle, the captured image obtained through a camera on the smart phone.
18. The assisted driving system as in claim 16 wherein the vehicle user's smart phone is configured to unlock the vehicle and allow the vehicle to be operated.
19. A non-transitory machine-readable medium storing executable computer program instructions which when executed by one or more data processing systems cause the one or more data processing systems to perform a method comprising: receiving object data representing an object size of an object placed on a vehicle; storing the object data for use by a data processing system coupled to the vehicle; capturing a first set of data using a set of one or more sensors on the vehicle, the set of one or more sensors coupled to the data processing system to provide data from which the data processing system is configured to determine a size of a space in an expected path of travel of the vehicle; comparing, by the data processing system, the size of the space to the object size to determine whether the vehicle, with the object placed on the vehicle, can pass through the space.
20. The medium as in claim 19, wherein the method further comprises: generating, through a user interface, a warning to the vehicle's user that the vehicle will not fit in response to determining, from the comparing, that the vehicle will not fit.
21. The medium as in claim 20 wherein the first set of data comprises one or more images and the set of one or more sensors includes one or more cameras and wherein the method further comprises: causing the vehicle to stop or change its path if the vehicle continues on the path toward the space after the warning is generated.
22. The medium as in claim 20 wherein the object data is received through a mobile application on the vehicle user's smart phone and the mobile application is configured to provide the object data to the data processing system through either a wired connection or a wireless connection and wherein the vehicle's user enters the object data through a user interface on the mobile application.
23. The medium as in claim 22 wherein the vehicle user's smart phone is configured to unlock the vehicle and allow the vehicle to be operated.
24. A non-transitory machine-readable medium storing executable computer program instructions which when executed by a data processing system cause the data processing system to perform a method comprising: storing data representing a vehicle size of a vehicle, the data representing the vehicle size being stored in a non-volatile memory on the vehicle, the non-volatile memory coupled to a data processing system; capturing a first set of data using a set of one or more sensors on the vehicle, the set of one or more sensors coupled to the data processing system to provide data from which the data processing system is configured to determine a size of a space in an expected path of travel of the vehicle; comparing the size of the space to the vehicle size to determine whether the vehicle can pass through the space.
25. The medium as in claim 24, wherein the method further comprises: generating, through a user interface, a warning to the vehicle's user that the vehicle may not pass through the space in response to determining, from the comparing, that the vehicle may not pass through the space.
26. The medium as in claim 25, wherein the method further comprises: causing, through control of a braking system or a steering system by the data processing system, the vehicle to stop or change its path if the vehicle continues on the path toward the space after the warning is generated.
27. The medium as in claim 26 wherein the vehicle size includes a tolerance value that is based on an uncertainty or range of possible errors in the set of one or more sensors that provide data used to determine the size of the space.
28. The medium as in claim 25 wherein the data processing system is coupled to one or more braking systems and one or more steering systems and to one or more motors to autonomously drive the vehicle or to assist in driving the vehicle and wherein the data processing system provides at least driving assistance if the vehicle continues on the path toward the space after the warning is generated, and the driving assistance guides the vehicle through the space if it can fit through the space.
29. The medium as in claim 28 wherein the warning includes a user interface to allow the vehicle's user to invoke driving assistance for guiding the vehicle through the space if possible.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0010] The present embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings in which like references indicate similar elements.
[0011]
[0012]
[0013]
[0014]
[0015]
[0016]
[0017]
DETAILED DESCRIPTION
[0018] Various embodiments and aspects will be described with reference to details discussed below, and the accompanying drawings will illustrate the various embodiments. The following description and drawings are illustrative and are not to be construed as limiting. Numerous specific details are described to provide a thorough understanding of various embodiments. However, in certain instances, well-known or conventional details are not described in order to provide a concise discussion of embodiments.
[0019] Reference in the specification to one embodiment or an embodiment means that a particular feature, structure, or characteristic described in conjunction with the embodiment can be included in at least one embodiment. The appearances of the phrase in one embodiment in various places in the specification do not necessarily all refer to the same embodiment. The processes depicted in the figures that follow are performed by processing logic that comprises hardware (e.g. circuitry, dedicated logic, etc.), software, or a combination of both. Although the processes are described below in terms of some sequential operations, it should be appreciated that some of the operations described may be performed in a different order. Moreover, some operations may be performed in parallel rather than sequentially.
[0020]
[0021]
[0022] The processing systems 29 can be a set of one or more processing systems which receive the object data and which uses the object data in conjunction with data from sensors 31 to determine whether the vehicle with the object on the vehicle can pass through a restricted space, such as an entrance to a garage or a tunnel, etc. The interface 27 can receive the object data representing the size of the object which is attached to the vehicle and provide that object data to the processing systems 29 which can then store the object data in memory 33. The object data 37 in memory 33 can be used during operation (e.g. movement) of the vehicle by the one or more processing systems such as the processing system 29. The processing system 29 can be coupled to one or more sensors 31 which can be a set of one or more cameras, one or more ultrasonic sensors, one or more radar sensors, and a LiDAR sensor. In one embodiment, all such sensors can be used together to provide a level of accuracy and redundancy that may not be possible with just one type of sensor. The one or more cameras can be used to capture images from which the size of a restricted space, such as the entrance of a garage, can be calculated. The size can be calculated using a reference distance known to the data processing system, such as the length of the front hood of a vehicle. The one or more cameras can provide image data while the radar systems, such as radar sensors which can be part of sensors 31, can provide distance information and the LiDAR sensor can provide distance information as well as image information in one embodiment. The processing system 29 can be configured to process the data from the sensors to determine the size of a restricted space and distance and compare that size to the size represented by the object data 37 stored in memory 33 in one embodiment.
[0023] When the processing system determines that the size of the object on the vehicle is larger than the height of the restricted space or larger than another measurement of the restricted space then the processing system can cause a warning to be displayed on an output, such as the output 35 which can be a display device or a set of speakers in the passenger compartment of the vehicle, etc. The warning can indicate to the driver that the car will not or may not safely pass through or fit through the restricted space which is in the expected path of the vehicle based upon the vehicle's current trajectory. In one embodiment, the object data 37 may include a tolerance value or error budget in order to accommodate mistakes in measurements. For example, the object data 37 may include a larger amount (such as 3% or 6% larger) for a measurement than the actual measurements supplied by the driver. In this example, if the actual measurement was 100 mm (supplied by the driver) and the tolerance value is based on 3%, then the object data may be set at 103 mm. In one embodiment, the tolerance value can be based upon a known level of uncertainty or range of possible errors in the one or more sensors that are used to measure the restricted space. In one embodiment, the processing system 29 can be coupled to one or more motors which are coupled to the wheels of the vehicle and can also be coupled to one or more braking systems and one or more steering systems which are coupled to the wheels of the vehicle. The processing system 29 in one embodiment can control the motors and steering system and the braking system to cause the vehicle to stop if the driver does not heed the warning that the vehicle will not safely pass through the restricted space. In one embodiment, the distance to the restricted space and the vehicle's current speed can be used to determine whether to intervene and stop the vehicle by using an autonomous driving system to control the motors and brakes to stop the vehicle.
[0024] The processing systems described herein, such as processing system 29 and processing systems 201, can provide in one embodiment assisted driving functionality, in which the vehicle assists the vehicle's user/driver, or autonomous driving, in which the vehicle controls steering, braking and movement through one or more motors without requiring input from the vehicle's user/driver. In one embodiment, a warning message that the vehicle may not be able to pass through a space, such as an entrance into a garage, can include a selectable user interface element that the vehicle's user can select to invoke an assisted driving functionality or autonomous driving to attempt to guide the vehicle through the space. The warning message and the selectable user interface element can be displayed on a touchscreen in the vehicle, and the vehicle's user can select the selectable user interface element by touching the element to invoke either an assisted driving functionality or autonomous driving. Once invoked, either the assisted driving functionality or autonomous driving can attempt to guide or drive the vehicle through the space if possible by controlling steering, brakes and motor(s) to attempt to guide the vehicle through the space using data from the one or more sensors, such as one or more cameras, a LiDAR sensor, a radar sensor, and ultrasonic sensors. The data processing system(s) in the vehicle can use the data from the one or more sensors to determine the clearance of different portions of the vehicle (e.g. right side, left side and top of vehicle) relative to the boundaries of the space (such as the door frame of a garage door), and this data about the clearance can be used to steer the vehicle in a manner to safely pass through the space (if possible) even if the space is a tight fit. The steering, under assisted driving or autonomous driving, can attempt to center the vehicle within a portion of the perceived space that appears to have enough clearance on all sides of the vehicle. In one embodiment, this centering operation can be performed iteratively in a process in which the data processing systems in the vehicle move the vehicle and concurrently measure the clearances and then adjust the steering to maintain sufficient and similar clearance on all sides and then again move and measure and then adjust the steering to maintain sufficient and similar clearance around the vehicle. If, during this process as the vehicle approaches the space, a clearance falls to nearly zero (or within a measurement tolerance error of zero) or below zero, then the data processing system can stop the vehicle before reaching the space and abandon the attempt to guide the vehicle through the space; moreover, in one embodiment, the vehicle's user can stop the vehicle by manually stepping on a brake pedal in the vehicle even if the vehicle is using autonomous driving to guide the vehicle. In one embodiment, the data processing system(s) of the vehicle can automatically invoke guidance through the space without displaying a selectable user interface element, and this automatic invoking of guidance can occur whenever the measured clearance around different portions of the vehicle falls below a threshold value.
[0025]
[0026]
[0027]
[0028] The LiDAR sensor 205 and the set of imaging radar sensors 207 can provide information with respect to the distance remaining to the restricted space which can be used by the processing system to determine when it should take action if the driver fails to heed the warning that the vehicle cannot pass through the restricted space. Further, ranging data from the LiDAR sensor 205 and ranging data from the one or more imaging radar sensors 207 (and optionally forward looking ultrasonic sensors) can be used to provide distance information that can be used to calculate the sizes of the restricted space based upon the distance. As is known in the art, the further away the restricted space is relative to the vehicle, the smaller it will appear in the image, and thus the distances provided by the ranging data can be used when the processing system(s) 201 calculates the size of the width and height of the restricted space, such as the restricted space 153. The assisted driving system 200 can also include a rearward looking sensor 213 (such as a camera). In one embodiment, the processing systems 201 can be coupled to one or more motors and a steering system and one or more braking systems on the vehicle. The one or more motors and a steering system and the one or more braking systems 211 can be coupled to wheels to allow the vehicle to move along a road. The processing system(s) 201 can control the motors and the braking systems to cause the vehicle to stop before attempting to pass through a restricted space in one embodiment. The vehicle can be a conventional gasoline powered vehicle or can be a battery powered vehicle or a hybrid vehicle that uses both gasoline and battery power to provide energy to motors to move the vehicle.
[0029] In one embodiment, the forward-looking camera, such as forward-looking camera 203 or forward-looking camera 107 can provide an angle of a ramp of the road in the expected path of the vehicle. This angle or slope can be used by a data processing system when calculating whether there is sufficient space to pass through or fit through a restricted space, such as restricted space 153. The angle data from the forward-looking camera can be used in combination with the calculated size of the restricted space to determine whether there is sufficient vertical clearance for the vehicle to continue along its expected path through the restricted space.
[0030] An example of a method, which can use the embodiments of the assisted driving systems described herein, will now be provided while referring to
[0031] In operation 311, the one or more processing systems can determine, from the camera data and/or data from other sensors the height and width of the clearance of a restricted space such as the height and width of a garage entrance. Then in operation 313, the one or more processing systems can compare the size of the clearance to the size of the vehicle with the attached object to determine whether there is sufficient clearance for the vehicle and the object to safely pass through or fit through the restricted space. If the processing system determines there may not be enough clearance, then the processing system will decide in operation 315 that a warning should be provided to the driver. If the clearance is sufficient, the processing system returns to operation 309 to continue cycling through operations 309, 311, 313, and 315. If the warning is presented as a result of operation 315, the data processing system can further perform operation 317 which can be optional in one embodiment. Operation 317 can involve the activation of autonomous driving which can include automatically stopping the vehicle if the warning is not heeded or attempting to pass through the space by guiding the vehicle through the space (if possible) by using autonomous driving.
[0032] If the vehicle includes autonomous driving capabilities, the activation of autonomous driving in one embodiment can be either immediate (after operation 313) or delayed if there is sufficient time to allow the driver to respond to the warning by stopping the vehicle. The data processing system(s) in the vehicle can use the vehicle's current speed and the distance to the restricted space to determine when the vehicle will enter the restricted space; for example, if the vehicle's current speed is 20 feet per speed and the restricted space is about 80 feet in front of the vehicle, the data processing system(s) have at least three seconds before potential impact, and may allow the driver one or two seconds before activating autonomous driving (to stop the vehicle or attempt to guide the vehicle through the space under the control of autonomous driving).
[0033] While this description has focused on objects attached to the vehicle, in other embodiments, the methods and systems described herein can be used for the vehicle itself without any objects attached to the vehicle, and thus the assisted driving system can determine if the vehicle (by itself) is too wide or too high for a restricted space. In this other embodiment, the vehicle can store data about the vehicle size (such as width, height and length), and use that data in a comparison to data about the size of a space in the path of travel of the vehicle. When that comparison indicates a tight fit, a warning can then be presented as described herein and assisted driving guidance or autonomous driving can be invoked (either manually or automatically).
[0034] While this description has focused on the use of different measurements, such as the width and height of the vehicle and the width and height of the space, it will be appreciated that other measurements or other approaches can be used in other embodiments. For example, a size matching algorithm that matches images of the same scale (in terms of pixels in the image per meter or other measurement unit of the object in the image) may be used to compare an image of the space to an image (or representation of an image) of the vehicle. The match in effect compares the sizes of objects in each image at the same scale to determine whether the vehicle can fit through the space. The image of the vehicle can be taken and stored by the vehicle's manufacturer for use in this size matching algorithm.
[0035] It will be apparent from this description that embodiments and aspects of the present invention may be embodied, at least in part, in software. That is, the techniques and methods may be carried out in a data processing system or set of data processing systems in response to one or more processors executing a sequence of instructions stored in a storage medium, such as a non-transitory machine readable storage media, such as volatile DRAM or nonvolatile flash memory. In various embodiments, hardwired circuitry may be used in combination with software instructions to implement the embodiments described herein. Thus the techniques and methods are not limited to any specific combination of hardware circuitry and software, or to any particular source for the instructions executed by the one or more data processing systems.
[0036] In the foregoing specification, specific exemplary embodiments have been described. It will be evident that various modifications may be made to those embodiments without departing from the broader spirit and scope set forth in the following claims. The specification and drawings are, accordingly, to be regarded in an illustrative sense rather than a restrictive sense.