IN-VEHICLE INFOTAINMENT SYSTEM COMMUNICATING WITH UNMANNED AERIAL VEHICLE AND METHOD OF OPERATING THE SAME
20200020231 ยท 2020-01-16
Assignee
Inventors
Cpc classification
B64U2101/20
PERFORMING OPERATIONS; TRANSPORTING
B60K2360/5915
PERFORMING OPERATIONS; TRANSPORTING
G06V20/588
PHYSICS
G08G1/166
PHYSICS
G06V20/56
PHYSICS
B64U2201/104
PERFORMING OPERATIONS; TRANSPORTING
B60K35/85
PERFORMING OPERATIONS; TRANSPORTING
B64U2101/30
PERFORMING OPERATIONS; TRANSPORTING
G08G1/167
PHYSICS
B64C39/024
PERFORMING OPERATIONS; TRANSPORTING
B60K35/00
PERFORMING OPERATIONS; TRANSPORTING
B60K35/28
PERFORMING OPERATIONS; TRANSPORTING
International classification
G08G1/0967
PHYSICS
B60K35/00
PERFORMING OPERATIONS; TRANSPORTING
Abstract
A method performed by an in-vehicle infotainment (IVI) system of a vehicle includes identifying an out-of-sight view of the vehicle, determining that information related to the identified out-of-sight view needs to be received from an unmanned aerial vehicle (UAV), identifying coordinates on the identified out-of-sight view to locate the UAV, based on an environment condition at a current position and at a future position of the vehicle, transmitting a signal indicating the coordinates to the UAV, and receiving the information related to the identified out-of-sight view from the UAV located at the coordinates.
Claims
1. A method performed by an in-vehicle infotainment (IVI) system of a vehicle, the method comprising: identifying an out-of-sight view of the vehicle; determining that information related to the identified out-of-sight view needs to be received from an unmanned aerial vehicle (UAV); identifying coordinates on the identified out-of-sight view to locate the UAV, based on an environment condition at a current position and at a future position of the vehicle; transmitting a signal indicating the coordinates to the UAV; and receiving the information related to the identified out-of-sight view from the UAV located at the coordinates.
2. The method of claim 1, wherein the signal indicating the coordinates comprises at least one of a latitude, a longitude, an elevation, an altitude, a mutual distance between a position of the vehicle and the coordinates, or a direction indicating the coordinates from the vehicle.
3. The method of claim 1, further comprising transmitting, to the UAV, updated information related to the vehicle for updating the position of the UAV.
4. The method of claim 1, wherein the UAV is docked onto the vehicle.
5. The method of claim 1, wherein the UAV is undocked from the vehicle.
6. The method of claim 1, wherein the out-of-sight view is identified based on at least one of traffic information, an environment condition, line-of-sight information, global positioning system (GPS) information, or a road condition.
7. The method of claim 1, wherein the environment condition comprises at least one of terrain, visibility information, weather information, an altitude, a position, or vehicle information.
8. The method of claim 7, wherein the environment condition is identified based on at least one of: an image of a surrounding area captured by the UAV, Internet or other sensors, a distance of turn, a distance of another vehicle in a path of travel, a distance between an object or position of interest and the vehicle, or a speed of an object or vehicle approaching in a path of travel, and wherein at least one of the distance of turn, the distance of the other vehicle in the path of travel, or the distance between the object or position of interest and the vehicle is identified based on the GPS information.
9. The method of claim 1, wherein the future position of the vehicle is identified based on at least one of braking performance of the vehicle, a current speed of the vehicle, GPS coordinates of the vehicle, or the position of the UAV.
10. A method performed by an unmanned aerial vehicle (UAV) communicating with a vehicle, the method comprising: receiving a signal indicating coordinates on an out-of-sight view of the vehicle from an in-vehicle infotainment (IVI) system of the vehicle; moving to the coordinates; identifying information related to the out-of-sight view of the vehicle; and transmitting the information related to the out-of-sight view of the vehicle to the IVI system.
11. The method of claim 10, further comprising: receiving updated information related to the vehicle from the IVI system; identifying updated coordinates based on the updated information related to the vehicle; and moving to the updated coordinates.
12. An in-vehicle infotainment (IVI) system of a vehicle, the IVI system comprising: at least one processor; and a communication module, wherein the at least one processor is configured to: identify an out-of-sight view of the vehicle; determine that information related to the identified out-of-sight view needs to be received from an unmanned aerial vehicle (UAV); identify coordinates on the identified out-of-sight view for locating the UAV, based on an environment condition at a current position and at a future position of the vehicle; transmit a signal indicating the coordinates to the UAV through the communication module; and receive the information related to the identified out-of-sight view from the UAV located at the coordinates through the communication module.
13. The IVI system of claim 12, wherein the at least one processor is further configured to transmit, to the UAV, updated information related to the vehicle for updating the position of the UAV, through the communication module.
14. The IVI system of claim 12, wherein the out-of-sight view is identified based on at least one of traffic information, an environment condition, line-of-sight information, global positioning system (GPS) information, or a road condition.
15. An unmanned aerial vehicle (UAV) communicating with a vehicle, the UAV comprising: at least one processor; and a communication module, wherein the at least one processor is configured to: receive a signal indicating coordinates on an out-of-sight view of the vehicle from an in-vehicle infotainment (IVI) system of the vehicle, through the communication module; control the UAV to move to the coordinates; identify information related to the out-of-sight view of the vehicle; and transmit the information related to the out-of-sight view of the vehicle to the IVI system, through the communication module.
16. The UAV of claim 15, wherein the at least one processor is further configured to: receive updated information related to the vehicle from the IVI system, through the communication module; identify updated coordinates based on the updated information related to the vehicle; and control the UAV to move to the updated coordinates.
17. The UAV of claim 15, wherein the UAV is docked onto the vehicle.
18. The UAV of claim 15, wherein the UAV is undocked from the vehicle.
19. A method performed by a primary vehicle communicating with at least one allied secondary vehicle, the method comprising: receiving, from the at least one secondary vehicle, a position of a road obstacle that is out-of-sight from the primary vehicle, the position of the road obstacle being identified by the at least one secondary vehicle; and identifying a position and a speed of the primary vehicle for preventing an accident, based on the position of the road obstacle, the position of the road obstacle being received from the at least one secondary vehicle.
20. A method performed by an in-vehicle infotainment (IVI) system of a first vehicle, the method comprising: identifying out-of-sight view information of the first vehicle; transmitting a connection request to an IVI system of a second vehicle; transmitting line-of-sight view information of the first vehicle to the IVI system of the second vehicle, after establishment of a connection with the IVI system of the second vehicle; receiving line-of-sight view information of the second vehicle from the IVI system of the second vehicle; identifying the out-of-sight view information of the first vehicle, based on the line-of-sight view information of the second vehicle and the line-of-sight view information of the first vehicle; and identifying a navigation path for the first vehicle, based on the out-of-sight view information of the first vehicle.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0014] The above and other aspects, features, and advantages of certain embodiments of the present disclosure will be more apparent from the following description taken in conjunction with the accompanying drawings in which:
[0015]
[0016]
[0017]
[0018]
[0019]
[0020]
[0021]
[0022]
[0023]
[0024]
[0025]
[0026]
[0027]
[0028]
DETAILED DESCRIPTION
[0029] Various embodiments of the present disclosure are described with reference to the accompanying drawings. However, various embodiments of the present disclosure are not limited to particular embodiments, and it should be understood that modifications, equivalents, and/or alternatives of the embodiments described herein can be variously made. With regard to description of drawings, similar components may be marked by similar reference numerals.
[0030] The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of exemplary embodiments of the disclosure as defined by the claims and their equivalents. It includes various specific details to assist in that understanding but these are to be regarded as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the disclosure. In addition, descriptions of well-known functions and constructions are omitted for clarity and conciseness.
[0031] The terms and words used in the following description and claims are not limited to their dictionary meanings but are merely used to enable a clear and consistent understanding of the disclosure. Accordingly, it should be apparent to those skilled in the art that the following description of embodiments of the present disclosure are provided for illustration purposes only and not for the purpose of limiting the disclosure as defined by the appended claims and their equivalents.
[0032] It is to be understood that the singular forms a, an, and the include plural referents unless the context clearly dictates otherwise.
[0033] The disclosure is directed to a road safety method to provide early warning and guidance to prevent accidents and avoid accident prone road behavior. The disclosure is applicable for both manual and driverless/autonomous vehicles.
[0034]
[0035]
[0036] Referring to
[0037]
[0038] At step 310, at least one out-of-sight view by an IVI of the vehicle is detected. The detecting of at least one out-of-sight view by an IVI of the vehicle is by a GPS which provides the terrain information for early computing of turns and of geographical areas that are in view or out of view with respect to the driver or the autonomous vehicle. In another embodiment, the IVI detects an overtaking scenario (i.e., passing another car), reverse movement or upcoming road turns using GPS or high resolution cameras installed in vehicle and computes undock command. The IVI may compute an undock command for the UAV based on, for example, a determination that the vehicle is put in reverse, the vehicle turns, or if the vehicle attempts to overtake (i.e., pass) another vehicle. The UAV may compute an undock command for itself based on, for example, a determination that the vehicle is put in reverse, the vehicle turns, or the vehicle attempts to overtake another vehicle. At step 320, the method checks that the detected out-of-sight view requires any assistance of the UAV by the IVI. In an embodiment, the UAV may or may not be mounted on the vehicle. The out-of-sight view is calculated based on at least one of but not limited to traffic information, environment condition, line of sight information, GPS information and road condition.
[0039] An operation of determining whether an out-of-sight view requires support of the UAV may be performed by the UAV. The UAV may recognize a sign board indicating a sharp curve by using a camera embedded in the UAV and transmit a request for GPS information regarding an incoming curve to the IVI in response to the identification of the sign board indicating the sharp curve. The UAV may recognize surroundings of the curve by using a situation recognition algorithm and identify the curve as out-of-view or as in-view.
[0040] At step 330, the future position of the vehicle is spotted by the UAV as not in the line of sight of the vehicle or driver. The calculation of the future position of the vehicle is based on at least but not limited to braking performance of the vehicle, current speed of the vehicle, GPS coordinates, elevation of current position coordinates and the position of the UAV.
[0041] At step 340, the method calculates the position of UAV based on environment condition at the current position and future position of the vehicle by the IVI or UAV. The position of the UAV may be computed by the IVI, and the position of the UAV may be computed by the UAV. The environment condition includes but is not limited to terrain, visibility information, weather information, altitude, position, and vehicle information. Further, the environment condition is derived from the captured image of the surrounding area near the UAV, the Internet or other sensors, a distance of a turn, a distance of another vehicle in the path of travel, a distance of an object or position of interest, and a speed of an approaching object or vehicle in the path of travel, wherein the distance information is calculated from GPS information.
[0042] At step 350, the UAV is positioned to facilitate navigation assistance on the detected out-of-sight view. The positioning of the UAV is based on at least one of coordinates, latitude, longitude, elevation, altitude, mutual distance and orientation. The UAV may be positioned at the destination coordinates to provide real time data from the position. The real time data is processed by the IVI and controls the speed and other safety operations of the vehicle at turns, while overtaking (i.e. passing) other vehicles, in reverse mode, along mountainsides and at varying elevations. In an embodiment, the IVI of the vehicle sends the coordinates (x, y, and z) or the latitude, longitude and altitude of a point to which UAV has to travel to and start assisting in navigation to avoid an accident by positioning itself to provide a better view of blind spots. The coordinates are calculated based on altitude, position, GPS, terrain, line of sight information, vehicle speed or other factors. In an embodiment, the coordinates of the UAV are based on the current position of the vehicle, GPS position of the turn, speed of the vehicle and terrain information.
[0043] The method re-positions the UAV with the updated information received from the vehicle and the vehicle position information. The re-positioning of the UAV is based on but is not limited to at least one of coordinates, latitude, longitude, elevation, altitude, mutual distance and orientation.
[0044]
[0045] Referring to
[0046]
[0047] Referring to
[0048]
[0049] Referring to
TABLE-US-00001 TABLE (1) Valid for Valid for Environment Detection Future Current S. No variable Parameter sensor Position Position 1 Obstacle Rocks, Camera, Yes No Trees, sensors signboards, water, ditch 2 Low Visibility Fog, smoke, Camera, Yes Yes direct sensor, sunlight, weather Night monitor application 3 Terrain Hills, Camera, Yes Yes buildings, GPS inclined roads, circular roads 4 U turn Hills, Camera, Yes Yes buildings, GPS inclined roads 5 Incoming/outgoing Vehicle Camera, Yes Yes vehicle sensors, (overtaking, sound line-of-sight) detection 6 Speed of Relative Camera, Yes Yes Incoming/Outgoing Speed of sensors, (overtaking, Vehicle vehicles sound turns) detection Note Some of the environmental parameters like vehicle, speed are directly used to calculate the position of UAV, whereas some other parameters helps in detecting future visible position from which position of UAV can be calculated.
[0050]
[0051] Once the UAV reaches the initial position, the UAV needs to get a clear view of the blind spot on the road. For example, the UAV will detect both of the vehicles with high resolution cameras (one camera pointing to a first vehicle and another camera pointing to a second vehicle). The UAV distance from each vehicle is calculated. For example, a distance from the first vehicle to the UAV and a distance from the second vehicle to the UAV is calculated. Based on these distances, the UAV is calibrated or self-adjusted.
[0052] To Get the Distance Z from a Vehicle to UAV Referring to
[0053] Z can be derived according to the following Equation (1):
[0054] where f is the focal length, T is the baseline and xrxl is the image disparity. The camera needs to be calibrated to find f in order to be able to obtain Z based on the image offset p (position of the object in the image from the axis) to the angle subtended by the object from the image center (refer to
[0055] The UAV, using the camera(s), obtains the distance of each object and re-positions itself based on the obtained distances.
[0056] Based on the decision variable D.sub.d, the following possibilities may arise as shown in Table (2):
TABLE-US-00002 TABLE (2) Decision variable Status New D.sub.d > Old D.sub.d There is no possibility of a crash between the two vehicles New D.sub.d < Old D.sub.d Vehicles moving towards each other; One vehicle is stationary; or One vehicle's speed is more than the other vehicle's speed in the same direction of movement New D.sub.d = Old D.sub.d Both the vehicles are stationary
[0057] The UAV repositions itself when the sight of view of either vehicle is lost and the same process will start again.
[0058] According to an embodiment, the coordinates of the UAV may be identified based on the following parameters as shown in Table (3).
TABLE-US-00003 TABLE (3) Current Position Coordinates of the UAV may be identified based on of Vehicle the current position of the vehicle. GPS Position of May be required for identification of initial Turning Point coordinates of the UAV. After the UAV is located at the initial coordinates, the position of the UAV may be adjusted to obtain a better view. Speed of Vehicle The view angle of the UAV may be identified based on the current speed of the vehicle. As the speed of the vehicle increases, the vehicle turns fast along a curve, thus the vehicle may need information about a far distance located on the opposite side of the turning point. Terrain The terrain information may be provided by the GPS. Information The terrain information may include a building, a hill, and other terrain information around the turning point.
[0059] When circular roads are consecutive, the GPS module may detect the turning point and transmit information about the turning point to a UAV controller. The UAV controller may determine whether the UAV is still in a flight mode by communicating with an IVI interface of the UAV. In this case, the UAV may be repositioned continuously instead of going back to the vehicle. The repositioning may be performed by transmitting, by the IVI, continuous coordinates to the UAV. Additionally, the repositioning may be performed by moving, by the UAV, to a new position each time the vehicle passes the turning point.
[0060] The UAV may start repositioning as the vehicle passes the UAV in a turn and the IVI using the GPS gets this information. The IVI may initiate the new coordinates for the UAV if a circular turn is determined to exist.
[0061] In addition, the UAV may start repositioning as the vehicle processes the images received from the UAV, and the IVI compares it with images of front camera of vehicle. If the image processing data is similar, then it may be determined that the UAV data is not directed to a blind spot for the vehicle.
[0062] Additionally, the UAV may start repositioning the vehicle as it passes it and is now monitoring the vehicle.
[0063] The repositioning as the vehicle passes the UAV in a turn and the IVI using the GPS gets the information is easily calculated by analyzing the angle 2 for the vehicle to which support is being provided, if the angle is decreasing from that of an old value then it suggests that the vehicle is going out of range and when the angle is zero or negative from that of the last value, then the UAV will trigger a re-positioning command to assist on the next turn. The angle here is only considered with respect to a connected vehicle (i.e., a following vehicle). The aforementioned characteristics of 2 are shown below in Table (4).
TABLE-US-00004 TABLE (4) Decision variable Status New 2 == Old 2 There is no possibility of crash between two vehicle as vehicle is in stationary state New 2 < Old 2 Vehicle moving forward New 2 == 0 or Both the vehicle passed the view angle of negative camera and now UAV need to re-position.
[0064] Additionally, the UAV may reposition itself based on a relative speed and a relative position of the vehicle according to the characteristics shown in Table (5), below.
TABLE-US-00005 TABLE (5) The speed of UAV can be compute based on following parameters - Speed of vehicle UAV gets the speed of vehicle as UAV has to travel faster than vehicle to reach the spot. Battery status of UAV If battery is low, then it may affect the speed of UAV and in that case IVI has to decide if trigger the undock command or not Distance to travel Travel distance decides the speed of UAV
[0065]
[0066] Referring to
[0067]
[0068]
[0069] At step 1010, each secondary vehicle using GPS, a camera or sensors locates the road blocks including incoming vehicle, potholes, or stoppers which are out-of-sight for the primary vehicle. The primary vehicle and the secondary vehicle are connected via the IVI system. The detecting of at least one out-of-sight view by an IVI of the vehicle is by a GPS which provides the terrain information for early computing of turns and a geographical area that are in view or out of view with respect to the driver/autonomous vehicle. The IVI may detect the overtaking (i.e., passing) scenario, reverse movement or upcoming road turns using GPS or high resolution cameras installed in the vehicle.
[0070] At step 1020, the position of the primary vehicle is calculated by a plurality of allied secondary vehicles.
[0071] At step 1030, all necessary information is received by the primary vehicle from the at least one secondary vehicle, where the primary vehicle and the secondary vehicle are connected with each other using the IVI of the vehicle.
[0072] At step 1040, all the data is processed by calculating the position and speed of the primary vehicle to take one or more precautionary measures.
[0073]
[0074] Referring to
[0075]
[0076] At step 1210, out-of-sight view information is detected by the IVI of the first vehicle based on various parameters including GPS data, camera data, and sensor information. The detecting of at least one out-of-sight view by an IVI of the vehicle is by a GPS which provides the terrain information for early computing of turns and geographical information for in view or out of view areas with respect to the driver or autonomous vehicle. Additionally, the IVI may detect the overtaking scenario (i.e., passing), reverse movement or upcoming road turns using GPS or high resolution cameras installed in vehicle.
[0077] At step 1220, the first vehicle is requested by the IVI to connect to the second vehicle, where the first vehicle and the second vehicle share the line-of-sight view information.
[0078] At step 1230, information on out-of-sight views of each vehicle is generated based on the shared line-of-sight view information between the two vehicles.
[0079] At step 1240, the line-of-sight view information of both the vehicles is combined to provide a complete view of the path for navigation in order to take one or more precautionary measures.
[0080]
[0081]
[0082] In the foregoing detailed description of embodiments of the disclosure, various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments of the disclosure require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby incorporated into the detailed description of embodiments of the disclosure, with each claim standing on its own as a separate embodiment.
[0083] While the present disclosure has been particularly shown and described with reference to certain embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the disclosure as defined by the appended claims and their equivalents.