System for parking a vehicle

Abstract

A system for parking a vehicle includes a mobile terminal. The mobile terminal includes at least one first image acquisition device for acquiring first image data, a first computing unit which is designed to receive the first image data from the first image acquisition device and to generate information indicating a parking position on the basis of the first image data, and a transmission device for transmitting the information. The vehicle includes a receiving device which is designed to receive the information, and a second computing unit which is designed to determine a parking strategy on the basis of the information. The vehicle includes a parking apparatus which is designed to park the vehicle in the parking position taking into account the parking strategy.

Claims

1. A system for parking a vehicle, comprising: a mobile terminal comprising (i) at least one first camera for capturing first image data, and (ii) an inertial sensor for delivering inertial data, the mobile terminal being configured to: (i) receive the first image data from the first camera; (ii) take the first image data as a basis for generating information indicating a parking position; (iii) transmit the information; and (iv) take the inertial data as a basis for determining an orientation and alignment of the mobile terminal; the vehicle comprising a second camera for capturing second image data, the vehicle being configured to: (i) receive the information; (ii) take the information as well as the inertial data or the orientation and alignment determined based thereon as a basis for determining a parking strategy; (iii) take into consideration the parking strategy to park the vehicle at the parking position; (iv) take a second image data from the second camera; (v) take the second image data as a basis for determining a relative position of the mobile terminal in relation to the vehicle, and (vi) take the relative position of the mobile terminal in relation to the vehicle as a further basis for determining a parking strategy.

2. The system as claimed in claim 1, wherein the information comprises a direction and a distance of the parking position relative to the mobile terminal.

3. The system as claimed in claim 1, wherein the mobile terminal is configured to determine a relative position and/or alignment of the vehicle in relation to the mobile terminal on the basis of the first image data.

4. The system as claimed in claim 1, wherein the mobile terminal, the vehicle and/or a computation unit of a server is configured to take the information as a basis for discerning a parking space for the vehicle.

5. The system as claimed in claim 1, wherein the mobile terminal is configured to take vehicle data and the information as a basis for providing a simulation of the parking process.

6. The system as claimed in claim 1, wherein the information comprises an absolute position of the parking space.

7. A method for parking a vehicle, the method comprising the steps of: capturing first image data on a camera of a mobile terminal; discerning a parking position on the basis of the first image data; transmitting information indicating a parking position to a vehicle; collecting inertial data of the mobile terminal; determining an orientation and alignment of the mobile terminal based on inertial data; capturing second image data on at least one second camera of the vehicle; determining the relative position of the vehicle in relation to the mobile terminal based on the second image data and the orientation and alignment of the mobile terminal; determining a parking strategy on the basis of the information transmitted by the mobile terminal and the relative position of the vehicle in relation to the mobile terminal; and parking the vehicle at the parking position using the parking strategy.

8. The method as claimed in claim 7, wherein the information comprises a direction and a distance relative to the mobile terminal.

9. The method as claimed in claim 8, wherein the information comprises an absolute position of the parking space.

10. The method as claimed in claim 7, wherein the information comprises an absolute position of the parking space.

11. The method as claimed in claim 7, the method further comprising the step of: determining a relative position of the vehicle in relation to the mobile terminal and/or an alignment of the vehicle relative to the mobile terminal on the basis of the first image data, wherein the relative position and/or the relative alignment of the vehicle are information.

12. A method for parking a vehicle configured to consider a parking strategy to park the vehicle in a parking position, the method comprising the steps of: receiving information indicating a parking position, said information being generated from first image data of a first camera of a mobile terminal and inertial data indicating the orientation and alignment of the mobile terminal, taking second image data captured on at least one second camera of the vehicle; determining, in the vehicle, a parking strategy based on the received information and the second image data; and parking the vehicle at the parking position using the determined parking strategy.

13. The method as claimed in claim 12, wherein the information comprises a direction and a distance relative to the mobile terminal.

14. The method as claimed in claim 13, wherein the information comprises an absolute position of the parking position.

15. A computer program product comprising a non-transitory computer-readable medium having stored thereon program code that, when executed by a processor, carry out a method to: capture first image data on a first camera of a mobile terminal; discern a parking position on the basis of the first image data; transmit information indicating a parking position to a vehicle; capture second image data on at least one second camera of the vehicle; determine orientation and alignment of the mobile terminal based on inertial data collected from an inertial sensor of the mobile terminal; determine a parking strategy on the basis of the transmitted information as well as the inertial data or the orientation and alignment determined based thereon; and park the vehicle at the parking position using the parking strategy.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 shows a vehicle having a multiplicity of image capture devices.

(2) FIG. 2 shows a schematic depiction of a mobile terminal.

(3) FIG. 3 shows a schematic depiction of the system in a first embodiment.

(4) FIG. 4 shows a schematic depiction of the system in a second embodiment.

(5) FIG. 5 shows a schematic depiction of the system in a third embodiment.

(6) FIG. 6 shows the geometric relationship between the system components and a parking position.

(7) FIG. 7 shows an exemplary parking situation in a first state.

(8) FIG. 8 shows an exemplary parking situation in a second state.

(9) FIG. 9 shows a first depiction of a parking process on a mobile terminal.

(10) FIG. 10 shows a second depiction of a parking process on a mobile terminal.

(11) In the description that follows, the same reference numerals are used for parts that are the same and have the same effect.

DETAILED DESCRIPTION OF THE DRAWINGS

(12) FIG. 1 shows a vehicle 10 having four image capture devices 11, 11, 11, 11, a parking apparatus 13, a reception device 14 and a computation unit 15. In the first embodiment of the vehicle 10, the image capture devices 11, 11, 11, 11 are RGB cameras for capturing image areas 12, 12, 12, 12. In further embodiments, the image capture devices 11, 11, 11, 11 may include radar units, LIDAR scanners, depth cameras, ultrasonic sensors or any other form of sensor system that is capable of detecting the surroundings of the vehicle. The parking apparatus 13 is configured to park the vehicle 10 in a discerned parking space autonomously. To this end, the parking apparatus 13 generates control signals that are sent to a control device (not shown) of the vehicle 10, so that the control device prompts the vehicle 10 to initiate applicable steering and acceleration or braking processes. The data of the image capture device 11, 11, 11, 11 of the vehicle 10 are forwarded to the computation unit 15 for processing. The computation unit 15 is, in the first embodiment, an onboard computer. The reception device 14 is configured to receive information 25 indicating a parking space wirelessly. This information 25 is provided by a mobile terminal 20.

(13) FIG. 2 shows a mobile terminal 20 of this kind in a highly simplified depiction. The mobile terminal 20 has an image capture device 21, a computation unit 22 and a transmission device 23. The image capture device 21 of the mobile terminal 20 is, in a first embodiment of the mobile terminal 20, an RGB camera. The mobile terminal 20 is a smartphone in the first embodiment. The captured image data 24 of the image capture device 21 of the mobile terminal 20 are forwarded for further processing to the computation unit 22 of the mobile terminal 20. After the processing, the information 25 is forwarded to the transmission device 23, which transmits the information 25 to the vehicle 10. This may firstly be a single transmission, on the other hand it is also possible for the transmission device to send data to the vehicle 10 continuously. Such continuous transmission of data has the advantage that the mobile terminal 20 itself needs to perform few computation operations. The computation unit 22 of the mobile terminal 20 may thus be of very low-spec design, which benefits the battery operating time of the mobile terminal 20. It is thus the case that the computation unit 22 of the mobile terminal 20 either processes the captured data of the image capture device 21 and/or forwards said data to the transmission device 23.

(14) FIG. 3 shows the system 1 for parking a vehicle 10 in a first embodiment. In the first embodiment, an image capture device 21 of the mobile terminal 20 captures image data 24 that are sent to a computation unit 22 of the mobile terminal 20. From the image data 24, the computation unit 22 ascertains information 25 indicating a parking position. In the first embodiment, the computation unit 22 detects a parking space in the image data 24.

(15) In the exemplary embodiment of FIG. 3, the computation unit 22 uses a Harris detector to detect features determining a parking space. In particular, the Harris detector can be used to determine the lines on a ground surface that has previously been extracted using the RANSAC algorithm. The arrangement of the features in the image allows a system trained in a training phase to identify a parking position 2 (see FIGS. 7 and 8).

(16) In further embodiments, the parking position 2 can additionally be validated. This means that a check is carried out to determine whether the vehicle 10 has enough space in the parking position. To this end, it is in particular possible for metadata about the vehicle 10 to be used, such as the height, the width and the length of the vehicle 10, for example. Based on the result of the validation, the parking position 2 can be used further. If the validation is negative, then the method can be restarted with a further possible position.

(17) The information 25 is now sent to a transmission device 23 of the mobile terminal 20 in order then to be sent from the transmission device 23 to the vehicle 10. In the first embodiment, this transmission is effected using a wireless network, e.g. based on the IEEE 802.11 standard. In other embodiments, e.g. Bluetooth, infrared or any other form of wireless transmission is also possible, however. Naturally, a wired transmission can also take place.

(18) The vehicle 10 receives the information 25 by way of a reception device 14. The reception device 14 subsequently sends the information 25 to a computation unit 15 of the vehicle 10. The computation unit 15 processes the information 25 further. On the basis of the information 25, the computation unit 15 ascertains a parking strategy 16 for the vehicle 10.

(19) The parking strategy 16 is, in the first embodiment, a vector V3 that points from the position of the vehicle 10 to the parking space 2 (see FIG. 6). In further embodiments, however, far more complex parking strategies are possible, such as e.g. a trajectory that has been ascertained by a movement planner. To ascertain the trajectory, a multiplicity of known algorithms can be considered, such as e.g. RRT (Rapidly Exploring Random Tree).

(20) The parking strategy 16 is subsequently transmitted to the parking apparatus 13. The parking apparatus 13 can then prompt the vehicle 10 to park in the parking space 2.

(21) FIG. 4 shows the system 1 in a second embodiment. In the second embodiment, the mobile terminal further includes an inertial sensor 26. The inertial sensor 26 is configured to deliver inertial data 27 to the computation unit 22 of the mobile terminal 20. The inertial data 27 comprise data that indicate the orientation and alignment of the mobile terminal 20. Together with the image data 24 of the image capture device 21 of the mobile terminal 20, the computation unit 22 can provide information 25 indicating a parking position 2, this information 25 comprising a relative distance and direction of the parking position 2 from the mobile terminal 20. The information 25 is then sent from the transmission device 23 of the mobile terminal 20 to the reception device 14 of the vehicle 10. The vehicle 10 includes, in a second embodiment, an image capture device 18 that sends image data 19 to the computation unit 15 of the vehicle 10. The computation unit 15 is therefore provided with the image data 19 and also the information 25 indicating a parking position 2. The computation unit 15 is configured to discern the mobile terminal 20 in the image data 19. It is therefore possible for the vehicle 10 to ascertain the position of the mobile terminal 20 relative to the vehicle 10. Using the information 25 indicating a position 2 at which the vehicle 10 is parked, it is possible for the computation unit 15 to determine the absolute position of the parking position 2. Such determination can be effected using simple means, such as triangulation. On the basis of the ascertained relative position of the parking position 2 of the vehicle 10, the computation unit 15 can compute a parking strategy 16. In the second embodiment, the parking strategy 16 indicates how the vehicle 10 is supposed to park in the parking position 2. This comprises a trajectory along which the vehicle 10 is supposed to travel. The parking apparatus 13, which is provided with the parking strategy 16 by the computation unit 15, prompts the vehicle 10 to follow the trajectory indicated in the parking strategy 16.

(22) FIG. 5 shows the system 1 in a third embodiment. In the third embodiment, the mobile terminal 20 sends information 25 to a server 30. In this case, the information 25 is sent to the server 30 as a continuous data stream. In the third embodiment, the information 25 merely includes captured image data 24 of the mobile terminal 20. The server 30 includes a third computation unit 31 that is configured to ascertain a parking position 2 from the information 25. The server 30 is, in the third embodiment, a web server that can be reached via the Internet. The transmission of the information 25 can therefore be effected from the mobile terminal 20 using a radio standard, such as e.g. LTE or UMTS. The computation unit 31 of the server 30 delivers the parking position 2 to the vehicle 10. The vehicle 10 can therefore, as in the previous embodiments, compute a parking strategy 16.

(23) FIG. 6 shows the geometric relationship of the vehicle 10, the mobile terminal 20 and the parking position 2 in a simplified, two-dimensional depiction. It can therefore be seen that if the mobile terminal 20 has discerned the parking position 2, a vector V1 can be computed that indicates the relative position of the parking position 2 in relation to the mobile terminal 20. If the vehicle 10 is configured to locate the mobile terminal 20 in a captured image area 12, 12, 12, 12, the vehicle 10 can determine a vector V2 that indicates the relative position of the mobile terminal 20 in relation to the vehicle 10. Using vector addition, it is therefore possible for a vector V3 to be determined that indicates the parking position 2 relative to the position of the vehicle 10. Using the vector V3, the vehicle 10 can ascertain the parking strategy 16 described previously. Naturally, it is possible for the relative positions to be provided in three-dimensional space. This means that it is also possible to discern those cases in which different parking positions are not in one plane.

(24) FIGS. 7 and 8 show an exemplary scenario in which a driver of a vehicle 10 initiates a parking process for the vehicle 10. FIG. 7 shows a mobile terminal 20 that is outside a vehicle 10. Two parked vehicles 3, 3 bound a parking space 2 in which the vehicle 10 can park. The driver first of all points the mobile terminal 20 at the vehicle 10, so that said vehicle is in a captured image area 12 of the mobile terminal 20. The mobile terminal 20 can then determine its position relative to the vehicle 10. Alternatively, the mobile terminal 20 may also be configured to receive a relative alignment and/or relative position of the vehicle 10 in relation to the mobile terminal 20.

(25) As time progresses, the driver can use a swivel movement to point the mobile terminal at the parking position 2. This is shown in FIG. 8.

(26) The mobile terminal 20 picks up the parked automobiles 3, 3 defining a parking position 2 and can therefore ascertain that this is a possible parking space for the vehicle 10. Further, the mobile terminal 20 can use the estimated height of the mobile terminal, as explained further above, to determine the distance and direction of the parking position 2. This information is subsequently sent by the mobile terminal 20 to the vehicle 10, as already described in connection with the other figures.

(27) Before the actual parking process for the vehicle 10, the mobile terminal 20 can provide the driver of the vehicle 10 with a simulation of the planned parking process. FIG. 9 shows a mobile terminal 20 that has a display 28. In the simulation shown in FIG. 9, the vehicle 10 is parked such that the front 17 of the vehicle 10, in the depiction of FIG. 9, points upward. By contrast, FIG. 10 shows a simulation in which the front 17 of the vehicle 10 points downward. The driver of the vehicle 10 is able, by means of an input on the mobile terminal 20, to determine the manner in which the vehicle 10 is supposed to travel into the parking position 2. To this end, the driver can make gestures on a touchscreen that symbolize a rotary movement. This is in particular advantageous if the driver wishes to have the trunk accessible, for example in order to pack shopping.

(28) In further embodiments, the simulation can be presented to the driver on a display and/or a smartglass by augmented reality. In this case, the simulation is overlaid on recorded images from the image capture device 21 of the mobile terminal 20. Therefore, the driver is provided with a very simple depiction of the complex parking situation. The parking situation can include the parking position 2 and/or the surrounding objects of the vehicle 10 or of a parking space and/or the vehicle geometry and/or geometry of vehicle components. The simulation can therefore also include a simulation of further vehicle components, such as the maximum opened position of doors and the trunk, for example. The driver can therefore discern whether sufficient space to open the doors is left and can take this into consideration directly when assessing the parking position 2.

(29) Optionally, the simulation provided allows a check by the driver to take place. If the driver establishes that the parking position 2 has been chosen unfavorably or even discerned incorrectly, he has the option of correcting it. In particular, the driver can perform a correction by selecting a further parking position 2. In a further embodiment, the selection of a further parking position 2 can be made automatically by the vehicle 10, by the mobile terminal 20 and/or by an external server 30.

(30) Additionally, the inputs of the driver can be transmitted to the vehicle 10. The vehicle 10 can subsequently take into consideration the transmitted inputs when ascertaining the parking strategy 16.

(31) In the embodiments described, it is possible for the mobile terminal to be either a smartphone, a smartwatch that has an image capture device or a smartglass. In particular, a smartglass has the advantage that the driver can scan the parking position 2 and prompt the vehicle 10 to park in the parking position 2 by merely looking at said parking position. Therefore, operation of the vehicle 10 is very simple.

LIST OF REFERENCE SYMBOLS

(32) 1 System 2 Parking position 3, 3 Parked vehicle 10 Vehicle 11, 11, 11, 11 Second image capture device 12, 12, 12, 12 Captured image area 13 Parking apparatus 14 Reception device 15 Second computation unit 16 Parking strategy 17 Front 18 Second image capture device 19 Second image data 20 Mobile terminal 21 First image capture device 22 First computation unit 23 Transmission device 24 First image data 25 Information 26 Inertial sensor 27 Inertial data 28 Display 30 Server 31 Third computation unit V1, V2, V3 Vector

(33) The foregoing disclosure has been set forth merely to illustrate the invention and is not intended to be limiting. Since modifications of the disclosed embodiments incorporating the spirit and substance of the invention may occur to persons skilled in the art, the invention should be construed to include everything within the scope of the appended claims and equivalents thereof.