METHOD, DEVICE, SERVER AND SYSTEM FOR CALIBRATING AT LEAST ONE CAMERA OF A DRIVER ASSISTANCE SYSTEM
20220191468 · 2022-06-16
Inventors
Cpc classification
G06T7/80
PHYSICS
G09G3/002
PHYSICS
H04N23/661
ELECTRICITY
International classification
H04N17/00
ELECTRICITY
B60R11/04
PERFORMING OPERATIONS; TRANSPORTING
G06T7/80
PHYSICS
G06V10/74
PHYSICS
G09G3/00
PHYSICS
Abstract
Disclosed herein is a method for calibrating at least one camera of a driver assistance system, comprising the steps of: imaging a calibration pattern using a display device and transmitting a control command to the camera to capture the calibration pattern imaged by the display device. Further disclosed is a method, a device, a server and a system (100) for calibrating at least one camera of a driver assistance system.
Claims
1. A method for calibrating at least one image sensor of a driver assistance system, the method comprising: imaging a calibration pattern by mans of using a display device located outside a vehicle; and transmitting a control command to the image sensor to capture the calibration pattern imaged by the display device.
2. The method according to claim 1, further comprising: requesting the calibration pattern, and obtaining the calibration pattern from at least one of a database or a memory.
3. The method according to claim 2, further comprising: at least one of inputting or transmitting vehicle data; and obtaining a vehicle-specific calibration pattern from the database.
4. The method according to claim 1, further comprising: determining at least one of a distance, a position, or an orientation of the display device in relation to the vehicle, modifying the calibration pattern depending on at least one of the distance, the position, or the orientation of the display device relative to the vehicle; and imaging the modified calibration pattern.
5. The method according to claim 4, further comprising: modifying at least one of a size, a position, or an orientation of the calibration pattern.
6. The method according to claim 1, wherein the display device includes at least one of: a display panel and a projector for projecting the calibration pattern onto the display panel; or a screen for imaging the calibration pattern.
7. The method according to claim 1, further comprising: obtaining at least one of an image signal comprising the calibration pattern or a file comprising the calibration pattern.
8. The method according to claim 1, further comprising: receiving, by the display device, the calibration pattern from an external component that is not part of the display device.
9. A method for calibrating at least one image sensor of a driver assistance system, the method comprising: transmitting a calibration pattern to a display device located outside a vehicle for imaging the calibration pattern; and transmitting a control command to the image sensor to capture the calibration pattern.
10. The method according to claim 9, further comprising: obtaining the calibration pattern captured by the image sensor; comparing the calibration pattern captured by the image sensor with the calibration pattern, determining, as a result of the comparison, at least one image sensor setting to be changed; and transmitting the at least one image sensor setting to be changed to the image sensor.
11. The method according to claim 9, further comprising: orienting or positioning the display device in relation to the vehicle.
12. The method according to claim 9, wherein the calibration pattern is transmitted by an external component to the display device, wherein the external component is not part of the display device.
13. The method according to claim 8, wherein the external component comprises at least one of a vehicle diagnostic device, a mobile terminal, or a server, or is formed by same.
14. The method according to claim 9, wherein the image sensor is a camera or comprises a camera.
15. A device for calibrating at least one image sensor of a driver assistance system, comprising: a display device located outside a vehicle for imaging a calibration pattern; and a first communication unit to transmit a control command to the image sensor to capture the calibration pattern imaged by the display device.
16. The device according to claim 15, wherein the display device comprises at least one of a screen for displaying the calibration pattern or a projector for projecting the calibration pattern.
17. The device according to claim 15, wherein the first communication unit is configurable to transmit the calibration pattern to the display device.
18. The device according to claim 15, further comprising: a memory with a first database which comprises two or more calibration patterns.
19. The device according to claim 15, further comprising at least one of: a sensor to determine at least one of a distance, or a position, or an orientation of the display device relative to the vehicle, a positioning component to position the display device in relation to the vehicle, or an orientation component for orienting the display device in relation to a vehicle.
20. The device according to claim 15, wherein the device is not part of the vehicle, and wherein the device is located outside the vehicle.
21. The device according to claim 15, wherein the display device is configurable to receive the calibration pattern from an external component that is not part of the display device.
22. The device according to claim 15, wherein the image sensor is a camera or comprises a camera.
23. The device according to claim 15, further comprising a server, the server including: a second communication unit; a second control and processing unit; and a second database; wherein the second database comprises two or more calibration patterns, wherein the second communication unit is configurable to transmit the calibration pattern to the display device and to transmit the control command to the image sensor.
Description
[0098] In the following, embodiments of the invention are explained in more detail with reference to the accompanying drawings. Here, the figures are schematised and partially simplified. They show:
[0099]
[0100]
[0101]
[0102]
[0103]
[0104]
[0105]
[0106] In the figures, recurring features are provided with the same reference signs.
[0107] First, reference is made to
[0108]
[0109] The projector 45 may comprise, for example, a light source and various optical elements, such as mirrors, lenses and/or apertures, for imaging the calibration pattern 1 on the display panel 44. In one embodiment, the projector 45 is designed as a retro-reflective projector. The display device 40 may also comprise at least one laser for imaging or projecting the calibration pattern. For example, the projector 45 comprises such a laser.
[0110] According to the illustration in
[0111] Furthermore, the chassis 12 may be equipped with four adjusting screws 5, each of which is arranged in a threaded bore of the longitudinal rails 13, 13 of the chassis 12, which are arranged parallel to one another, in such a way that they are intended to stand adjustably on the supporting substrate of the chassis 12. By means of these adjusting screws 5 the optical display unit 44, 45, 46 can be oriented in such a way that its upper edge is horizontal and its side edges are vertical. The correct orientation of the optical display unit 44, 45, 46 can be checked by means of an adjustment bar spirit level 6 arranged on the adjustment bar 4, since the adjustment bar 4 is also oriented horizontally.
[0112] The display panel 44 of
[0113] As a further aid for orienting and positioning the display panel 44 or the screen 46 in relation to the vehicle 60 or the vehicle leading axle or the unsteered vehicle axle, two wheel sensors 20 of identical design may be provided; see
[0114] The details of how the display device 40 with the display panel 44 or screen 46 can be oriented and positioned with respect to the vehicle 60 is disclosed, for example, in the publication DE 10 2015 112 368 A1. In DE 10 2015 112 368 A1 a calibration panel is oriented and positioned which is similar in form and function to the display panel 44 or screen 46 of the present application. In addition, DE 10 2015 112 368 A1 explains further details of the wheel sensors 20 and their functions.
[0115] Further, the display device 40 may include an orientation sensor 47 and/or a position sensor 48 and/or a distance sensor 49 (see
[0116] The orientation sensor 47 can be used to determine a relative or absolute orientation of the display device 40. The orientation sensor 48 may include, for example, an accelerometer, a gyroscope and a geomagnetic field sensor, each of which can provide accurate acceleration, angular velocity (gyroscopic) and geomagnetic field measurements in any spatial direction.
[0117] For determining the position of the display device 40 with respect to the vehicle 60 (left, centred or right), the position sensor 48 may be used, which comprises, for example, an inductive sensor, a capacitive sensor, a magnetic sensor, an ultrasonic sensor, an opto-electric sensor, a laser sensor, a distance sensor, a forked light barrier, an angular light barrier and/or a magnetic cylinder sensor. Similar sensors can also be used for the distance sensor 49. The distance sensor 49 is used to determine a distance of the display device 40 to the vehicle 60. Alternatively or additionally, a measuring tape may be used to determine the distance and/or position of the display device 40 relative to the vehicle. The orientation of the display device 40 can be carried out with the aid of at least one spirit level.
[0118] Thus, with the aid of the means described above, the means disclosed in DE 10 2015 112 368 A1 and/or further means, it can be ensured that the display device 40 can be positioned and oriented in front of a vehicle 60 in such a way that a calibration of a camera 65 of a driver assistance system 64 or of an image sensor of a driver assistance system 64 can be performed. In particular, the display device 40 can be positioned at a defined distance and centrally in front of the vehicle 60 and at right angles to the longitudinal axis of the vehicle.
[0119]
[0120] Some of the components 40, 50, 70, 80 shown in
[0121] First of all, we will start from
[0122] The diagnostic interface 66 is sometimes also called an OBD interface (on-board diagnostics interface). As a rule, the vehicle diagnostic interface 66 is located in a footwell of the vehicle 60 and is designed as a socket. The vehicle diagnostic interface 66 is connected to at least one vehicle control device 61 located in the vehicle 60 via corresponding signal lines. The vehicle control device 61 is in turn connected to at least one camera 65 of a driver assistance system 64, which is designed to capture images of an immediate environment of the vehicle 60. The at least one camera 65 may be arranged, for example, in or on a windscreen of the vehicle 60.
[0123] The operating principle of the driver assistance system 64, the camera 65 and the vehicle control device 61 and the communication between the driver assistance system 64 and the vehicle control device 61 are known from the prior art and are only briefly outlined below. The camera 65 can be designed to take pictures at regular intervals. The camera 65 may also be configured to take pictures after receiving a control command. The camera 65 usually receives the control command from the vehicle control device 61. The vehicle control device 61 may be configured to receive the control command from another unit and to forward it to the camera 65. The camera 65 can send the images at regular intervals to the vehicle control device 61, where they are subsequently evaluated. The digital images can also be forwarded to other units by the vehicle control device 61. The captured images can be temporarily stored or permanently stored in the memory 63 of the vehicle 60 designed for this purpose. It should also be noted that the operating principle of the driver assistance system 64, the camera 65 and the vehicle control device 61 is specified by the manufacturer of the driver assistance system and thus does not necessarily form part of the present invention.
[0124] Furthermore,
[0125] The display device 40 has already been described in conjunction with
[0126] Further,
[0127] As already sufficiently known from the prior art, the vehicle control device 50 can be designed to read out error codes (diagnostic trouble code, DTC) stored in the vehicle. After this, the error codes can be analysed by the vehicle diagnostic device 50 to diagnose whether and which vehicle components need to be repaired or replaced in order to correct the problem. Thus, by means of an evaluation of the error codes (vehicle diagnostics), a conclusion can be made as to which vehicle components are defective and require repair.
[0128] Typically, the communication unit 52 of the vehicle diagnostic device comprises a first transmitter and receiver unit (not shown), which is designed for wired communication with the vehicle control device 61. Further, the communication unit 52 may comprise, for example, a second transmitter and receiver unit (not shown) for wirelessly communicating with the display device 40 (or the communication unit 42 of the display device 40), which typically operates according to a previously known radio standard or radio protocol. The first transmitter and receiver unit and the second transmitter and receiver unit of the vehicle diagnostic device 50 may be connected to each other, either directly or via the control unit 51, such as a processor or a controller. The control unit 51 of the vehicle diagnostic device 50 may control a communication between the first transmitter and receiver unit and the second transmitter and receiver unit of the vehicle diagnostic device 50.
[0129] In particular, the communication unit 52 (or the second transmitter and receiver unit of the communication unit 52) may comprise a Bluetooth unit, a WLAN (for example according to IEEE 802.11) or another unit designed to communicate with the communication unit 42 of the display device 40 via a near-field connection. Other wireless connections such as EnOcean, Z-Wave, ZigBee, WiMAX, UMTS/HSDPA, LTE (long term evolution), NanoNetm, UWB (Ultra Wideband), NB-IoT (Narrowband Internet of Things), Sigfox or LoRa are also possible for the communication of the first communication unit and are known to a person skilled in the art. Alternatively, the vehicle diagnostic device 50 may communicate with the display device 40 by wire. The transmitter and receiver units 52, 42 are thus designed to communicate with each other wirelessly or by wire.
[0130] The communication unit 52 may be adapted to transmit a control command to the camera 65 to capture the calibration pattern 1 imaged by the display device 40. In addition, the communication unit 52 may be configured to transmit or forward the calibration pattern 1 to the display device 40.
[0131] In one embodiment, the memory 43 and/or the memory 53 comprise a database comprising a plurality of calibration patterns 1.
[0132]
[0133] The communication unit 72 may be designed to transmit the calibration pattern 1 to the display device 40. In addition, the communication unit 72 may be designed to transmit a control command to the camera 65 to capture the calibration pattern 1 imaged by the display device 40.
[0134] The communication unit 52 of the vehicle diagnostic device 50 may be further designed to communicate with the communication unit 72 of the server 70. The communication of the vehicle diagnostic device 50 with the server 70 may be by wire and/or wireless. A person skilled in the art is familiar with the fact that communication with the communication unit 72 of the server 70 can take place via intermediate network components, such as base stations and/or further servers.
[0135] The control unit 51 of the vehicle diagnostic device 50 may be designed to handle or to process received data from the server 70 and/or the vehicle control device 61. If necessary, data can be temporarily stored or buffered in a memory 53 designed for this purpose before being forwarded to another component. The memory 53 may store a program for controlling the electronic components of the vehicle diagnostic device 50.
[0136] The server 70 and the device 10 together form a system 100 which is suitable and/or designed in particular for calibrating the camera 65.
[0137] Instead of the vehicle diagnostic device 50, a vehicle communication interface (VCI, not shown) may also be connected to the vehicle diagnostic interface 66. The vehicle communication interface may be, for example, an adapter having a plug for mechanical and electrical connection to the vehicle diagnostic interface 66. Further, the adapter may comprise a communication unit for communicating with the communication unit 42 of the display device 40 and/or the communication unit 72 of the server. The adapter may in particular be designed to enable communication between the display device 40 and the vehicle control device 61 or between the server 70 and the vehicle control device 61.
[0138] Furthermore,
[0139] Typically, the communication unit 82 comprises a transmitter and receiver unit (not shown) for wirelessly communicating with the communication units 42, 52 and 72 or the aforementioned adapter, which transmitter and receiver unit typically operates according to a previously known radio standard or radio protocol. For possible wireless connections, refer to the connections mentioned in relation to communication unit 52 (see above). Additionally or alternatively, the communication of the communication unit 82 with the communication units 42, 52, 72 or the adapter may also take place by wire. The mobile terminal 80 is configured to receive, forward and/or process data from the vehicle diagnostic device 50 and/or the server 70 and/or the adapter.
[0140] The vehicle diagnostic device 50 is configured to receive, forward and/or process data from the vehicle control device 61, the camera 65, the driver assistance system 64, the server 70, the display device 40 and/or the mobile terminal 80. The display device 40 is configured to receive and/or process data from the vehicle diagnostic device 50 and/or the mobile terminal 80.
[0141] In the following, for the sake of simplicity, reference will mainly be made to the display device 40, the vehicle diagnostic device 50, the camera 65, the server 70 and the mobile terminal 80. However, it is clear that communication between these units is via the corresponding communication units 42, 52, 62, 72, 82 and that the control/processing of any data or signals is carried out by the corresponding control and processing units 41, 51, 61, 71, 81.
[0142] In particular, the device 10 and/or the system 100 of
[0143] First, the display device 40 is oriented and positioned in front of the vehicle 60 in such a way that a calibration of the camera 65 can be performed. Thereafter, the display device 40 positioned and oriented in front of the vehicle 60 displays the calibration pattern 1. A control command is then sent to the camera 65. The control command causes the camera 65 to record the calibration pattern 1 imaged by the display device 40. The control command may in particular be sent to the camera 65 by the communication unit 52 of the vehicle diagnostic device 50. In this case, the vehicle control device 61 receives the control command from the vehicle diagnostic unit 50 and forwards the control command to the camera 65. Alternatively, the control command is sent from the server 70 to the camera 65. In this case, the control command is sent from the server 70 to the vehicle diagnostic device 50 and from there to the vehicle control device 61, which in turn forwards it to the camera 65. The control command may also be sent to the camera by the mobile terminal 80 via the vehicle diagnostic device 50 and the vehicle control device 61.
[0144] After receiving the control command, the camera 65 records the calibration pattern 1 imaged by the display device 40. The images taken by the camera can be evaluated on the vehicle side by the control unit 61 in order to calibrate the camera 65. This allows the images from the camera 65 to be compared with a calibration pattern 1 stored in the memory 63 in the vehicle. The camera 65 is then calibrated based on the comparison.
[0145] Alternatively, the images captured by the camera 65 of the calibration pattern 1 imaged by the display device 40 are sent to the vehicle diagnostic device 50 and/or to the server 70, where the images from the camera 65 are then compared with the calibration pattern 1. For example, camera settings to be changed can be determined by the comparison. The camera settings to be changed are then sent to the vehicle control device 61 or the camera 65.
[0146] Prior to imaging, the display device 40 may request and receive the calibration pattern 1. For example, the vehicle diagnostic device 50 transmits the calibration pattern 1 stored in the memory 53 to the display device 40.
[0147] Alternatively, the display device 40 may retrieve the calibration pattern 1 from the memory 43. The calibration pattern 1 may also be obtained from the database 73 of the server 70 in a further embodiment. In this case, the vehicle diagnostic device 50 forwards the calibration pattern 1 to the display device 40. According to a variant, the mobile terminal 80 can also transmit the calibration pattern 1 to the display device 40.
[0148] In order for the display device 40 to obtain the correct calibration pattern 1, vehicle data, such as VIN, vehicle type, vehicle manufacturer and/or year of manufacture of the vehicle, may be transmitted to the components 50, 70, 80 beforehand. Thereafter, the calibration pattern 1 corresponding to the vehicle data is sent to the display device 40 by the corresponding components 50, 70, 80.
[0149] The display device 40 may receive the calibration pattern 1 as a file or image signal. Thus, the step of obtaining the calibration pattern 1 may comprise: [0150] obtaining an image signal comprising the calibration pattern 1 or [0151] obtaining a file comprising the calibration pattern 1.
[0152] Upon receipt of the calibration pattern 1, a confirmation of receipt may be sent to the component that transmitted the calibration pattern 1 to the display device 40. After receiving the confirmation, the corresponding component can transmit the control command to the camera 65. As a result, the camera 65 takes a picture of the calibration pattern 1 imaged by the display device 40.
[0153] The orientation, position and distance of the display device 40 can be determined with the aid of the orientation sensor 47, the position sensor 48 or the distance sensor 49 or the measuring tape. Since the calibration pattern 1 is electronic or digital, the sensor data from the sensors 47, 48, 49 can be used to process the calibration pattern 1 before it is displayed. In addition or alternatively, the distance, position and/or orientation of the display device 40 relative to the vehicle 60 may be manually input, for example, via one of the input and output units 54, 84. Position, distance and orientation could thus also be determined and input manually, for example using the measuring tape. The input unit may, for example, be part of the display device 40, the vehicle diagnostic device 50 and/or the mobile terminal 80. In this way, the calibration pattern 1 can be imaged depending on the distance, position (left/right) and orientation of the display device 40 relative to the vehicle 60. For example, a size, position and/or orientation of the calibration pattern 1 imaged by the display device 40 may be adjusted depending on the position and/or orientation of the display device 40 with respect to the vehicle 60. The display device 40 may modify the calibration pattern 1 to be imaged, such as shifting, rotating, reducing and/or enlarging it, and image the modified calibration pattern 1. Alternatively or additionally, the projector 45 may be rotated or shifted (to the left/right or to the rear/front) relative to the display wall 44 in order to modify the calibration pattern 1 imaged on the display wall 44. Instead of the projector 45, at least one optical component (such as lenses, mirrors, apertures, etc.) used in the projector 45 may also be shifted to modify the calibration pattern 1 projected onto the display panel 44.
[0154] Features mentioned only in relation to individual components, the device 10, the server 70 or the system 100 can also be claimed for the methods described and vice versa, provided they do not contradict each other.
LIST OF REFERENCE SIGNS
[0155] 1 Calibration pattern [0156] 2 Adjustment bar scale [0157] 3 Mirror [0158] 4 Adjustment bar [0159] 5 Adjusting screws [0160] 6 Spirit level [0161] 10 Calibration device [0162] 12 Chassis [0163] 13 Longitudinal rail [0164] 13′ Longitudinal rail [0165] 15 Support frame [0166] 20 Wheel sensor [0167] 21 Laser [0168] 40 Display device [0169] 41 Control and processing unit [0170] 42 Communication unit [0171] 43 Memory [0172] 44 Display panel [0173] 45 Projector [0174] 46 Screen [0175] 47 Orientation sensor [0176] 48 Position sensor [0177] 50 Vehicle diagnostic device [0178] 51 Control and processing unit [0179] 52 Communication unit [0180] 53 Memory [0181] 54 Input and output unit [0182] 60 Vehicle [0183] 61 Vehicle control device [0184] 62 Communication unit [0185] 63 Memory [0186] 64 Driver assistance system [0187] 65 Camera [0188] 66 Diagnostic interface [0189] 70 Server [0190] 71 Control and processing unit [0191] 72 Communication unit [0192] 73 Database [0193] 80 Mobile terminal [0194] 81 Control and processing unit [0195] 82 Communication unit [0196] 83 Memory [0197] 84 Input and output unit [0198] 100 System