Apparatus and method for checking the playback of a video sequence of a mirror replacement camera

Abstract

An apparatus for checking the playback of a video sequence of a mirror replacement camera for a motor vehicle includes: a movable camera device (10), which is designed to record an image and to provide image data; an image data processing device (20), which is coupled to the camera device (10) and is designed to produce the video sequence based on the captured image data; an output device (30), which is designed to display the produced video sequence; and a diagnosis device (40), which is designed to move the camera device (10) and/or to generate an image movement when producing the video sequence so as to check the displayed video sequence.

Claims

1. A system for a motor vehicle, the system comprising: a mirror replacement camera comprising a camera device, which is mounted on the motor vehicle and is configured to record images and to provide corresponding image data; an image data processing device, which is coupled to the camera device and is configured to produce a video sequence based on the image data; an output device, which is configured to display the video sequence; and a diagnosis device, which is configured to produce an image movement in the recorded images by moving the camera device or a lens thereof or by generating the image movement in the producing of the video sequence, so as to check whether the video sequence has frozen by evaluating the video sequence before and after the image movement; wherein the diagnosis device is configured to perform the check of the video sequence when an eye contact of an occupant of the motor vehicle with the output device has been captured.

2. The system according to claim 1, wherein the diagnosis device is configured to perform the check of the video sequence repeatedly at predetermined periods of time.

3. The system according to claim 1, wherein the diagnosis device is configured to perform the check of the video sequence dependent on a prevailing driving situation and/or a prevailing vehicle state.

4. The system according to claim 1, wherein the camera device is configured to move a field of view of the camera device so as to produce the image movement.

5. A method of monitoring a mirror replacement system for a motor vehicle, the method comprising the steps: recording images and providing corresponding image data by a camera device mounted on the motor vehicle; producing a video sequence based on the image data; displaying the video sequence; detecting whether an occupant of the motor vehicle makes an eye contact with the output device; and when the eye contact has been detected, checking whether the video sequence has frozen by producing an image movement in the recorded images by moving the camera device or a lens thereof or by generating the image movement in the producing of the video sequence, and evaluating the video sequence before and after the producing of the image movement.

6. The system according to claim 1, wherein the camera device is movable, and the diagnosis device is configured to produce the image movement by moving the camera device.

7. The system according to claim 1, wherein the diagnosis device is configured to produce the image movement by moving the lens of the camera device.

8. The system according to claim 1, wherein the diagnosis device is configured to produce the image movement by generating the image movement in the producing of the video sequence.

9. The system according to claim 1, wherein the diagnosis device is configured to perform the evaluating of the video sequence by comparing the video sequence after the image movement with the video sequence before the image movement to determine whether differences resulting from the image movement can be detected.

10. A method of monitoring a sensor output of a sensor system of a motor vehicle, comprising steps: a) with a sensor device of the motor vehicle, producing a succession of sensor outputs; b) with a data processor device of the motor vehicle, producing a data sequence based on the succession of sensor outputs; c) with a display device of the motor vehicle, displaying a display output based on the data sequence; d) detecting whether an occupant of the motor vehicle makes an eye contact with the display device; and e) when the eye contact has been detected, checking whether the data sequence produced by the data processor device has frozen, by changing an effective sensing direction of the sensor device, continuing the steps a) and b), and comparing the data sequence after the changing of the effective sensing direction with the data sequence before the changing of the effective sensing direction, to determine whether a difference is detected in the data sequence after as compared with before the changing of the effective sensing direction.

11. The method according to claim 10, wherein the sensor device comprises a camera, and the sensor outputs comprise camera images.

12. The method according to claim 10, wherein the sensor device comprises a radar sensor.

13. The method according to claim 10, wherein the sensor device comprises an ultrasound sensor.

14. The method according to claim 10, wherein the changing of the effective sensing direction comprises physically moving the sensor device relative to a vehicle body of the motor vehicle.

15. The method according to claim 10, wherein the changing of the effective sensing direction comprises moving a lens of the sensor device to a different orientation.

16. The method according to claim 10, wherein the changing of the effective sensing direction comprises redirecting a field of view of the sensor device.

17. The method according to claim 10, wherein the changing of the effective sensing direction comprises orienting a field of view of the sensor device to encompass a portion of a vehicle body of the motor vehicle.

18. The method according to claim 10, further comprising automatically remedying a freeze error upon determining that the data sequence produced by the data processor device has frozen.

Description

BRIEF DESCRIPTION OF THE FIGURES

(1) FIG. 1: shows a schematic representation of the entire system having a camera, also abbreviated to MRS for Mirror Replacement System, in order to explain the present invention;

(2) FIG. 2: shows a schematic representation of a mirror replacement camera in order to explain the present invention;

(3) FIG. 3: shows a schematic representation of a timeline in order to clarify the chronology of two images recorded in chronological succession according to an exemplary embodiment of the present invention;

(4) FIG. 4: shows a schematic representation of an exemplary modification of the camera lens having another viewing range according to another exemplary embodiment of the present invention;

(5) FIG. 5: shows a schematic representation of a flow chart of a method for checking the playback of a video sequence of a mirror replacement camera according to another exemplary embodiment of the present invention;

(6) FIG. 6: shows a schematic representation of a flow chart of a method for checking the playback of a video sequence of a mirror replacement camera according to another exemplary embodiment of the present invention;

(7) FIG. 7: shows a schematic representation of a flow chart of a method for checking the playback of a video sequence of a mirror replacement camera according to another exemplary embodiment of the present invention;

(8) FIG. 8: shows a schematic representation of a possibility of an installation of the mirror replacement camera according to another exemplary embodiment of the present invention;

(9) FIG. 9: shows a schematic representation of a flow chart of a method for checking the playback of a video sequence of a mirror replacement camera according to another exemplary embodiment of the present invention;

(10) FIG. 10: shows a schematic representation of an apparatus for checking the playback of a video sequence of a mirror replacement camera according to another exemplary embodiment of the present invention; and

(11) FIG. 11: shows a schematic representation of a flow chart of a method for checking the playback of a video sequence of a mirror replacement camera according to another exemplary embodiment of the present invention.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

(12) In the figures of the drawings, the same reference numerals denote elements, parts or components or method steps which are the same or which have the same function, unless otherwise indicated.

(13) The motor vehicle or respectively vehicle is, for example, a motor vehicle or a hybrid vehicle, for example a hybrid vehicle having a coasting function, for example a motorcycle, a bus or a truck or a bicycle.

(14) An apparatus for checking the playback of a video sequence of a mirror replacement camera can be used for further camera-based and sensor-based systems.

(15) In principle, the method can also be transferred to a radar or lidar sensor and the apparatus for checking the playback of a video sequence can be deployed during this.

(16) The sensor does not have to be movable as such since it can also be provided that the vehicle is utilized as such as a movable object, or a part of the vehicledoor, electric window, taillight, turn signal lightis utilized as a movable object depending on the scale of the recorded image.

(17) FIG. 1 shows a schematic representation of a description of the entire system having a camera, a Mirror Replacement System, also abbreviated to MRS, in order to explain the present invention.

(18) FIG. 1 shows a camera or a camera device which is installed in a vehicle and which has a specific viewing range or respectively a field of view.

(19) Additionally, an image data processing device, an MRS, and a display in the form of an output device for displaying the image data or respectively the video sequence are represented.

(20) FIG. 2 shows a schematic representation of a mirror replacement camera in order to explain the present invention.

(21) FIG. 2 shows an exemplary view of a mirror camera.

(22) A part or a partial region of the vehicle is generally also visibleor respectively such an environment should be visible so that the driver can better orient himself regarding what the camera is directed at.

(23) The represented method describes a system:

(24) i) which records various full images and compares these with one another over time; and/or

(25) ii) which automatically aligns or moves the camera and which compares the images with one another.

(26) The present invention proposes a mechanism which provides that the camera is moved during operation and while images are displayed on the screen so as to check the display and the entire system. Multiple cameras can additionally be actuated accordingly and checking movements can be carried out with said cameras.

(27) The camera or respectively the lens is, for example, moved to the left or right so as to record another scene. The camera thus focuses on new objects, even in the case of stationary scenery, that is to say when there is no movement of the vehicle or other objects in the recorded scenery.

(28) An image is therefore recorded which is different from the previous image which was recorded prior to the movement.

(29) For example, the camera can be aligned in the direction of the vehicle body as a background is always to be expected here.

(30) Said background can also be prestored and can also be clearly determined on the basis of the angle of the cameraexcept in extremely thick fog and heavy rain.

(31) FIG. 3 shows a schematic representation of a timeline in order to clarify the chronology of two images recorded in chronological succession according to an exemplary embodiment of the present invention.

(32) FIG. 3 shows a comparison of two consecutive images. This can also be a method so as to recognize a freezing of the image processing system or respectively an image loss of the image processing system.

(33) Images recorded consecutively in time by the camera system are to be stored and these consecutively recorded images are subsequently compared with one another in the computer system, using software-based image processing methods.

(34) FIG. 4 shows a schematic representation of an exemplary modification of the camera lens having another viewing range according to another exemplary embodiment of the present invention.

(35) FIG. 4 indicates, by way of example, that during the movement of the camera another field of viewwhich deviates from the field of view of the first imageis created.

(36) FIG. 5 shows a schematic representation of a flow chart of a method for checking the playback of a video sequence of a mirror replacement camera according to another exemplary embodiment of the present invention.

(37) FIG. 5 shows a possible difficulty during the simple comparison of images of recorded images over time.

(38) If, for example, a vehicle is to drive out of a parking place or parking space, it can happen that at the time of the recording, few or no moving objects at all can be recognized in the field of view of the camera.

(39) As shown in the diagram, the camera records images consecutively in time and compares these recorded images.

(40) In this case, the camera does not find any differenceit may be that the camera monitor system has frozen or not, i.e. there may not be any movement of the represented objects.

(41) FIG. 6 shows a schematic representation of a flow chart of a method for checking the playback of a video sequence of a mirror replacement camera according to another exemplary embodiment of the present invention.

(42) FIG. 7 shows possible triggering events, also called triggers, for triggering the method for checking the playback of a video sequence of a mirror replacement camera according to another exemplary embodiment of the present invention.

(43) FIG. 7 shows an embodiment of the present invention, in which recordings or images of specific parts or partial regions of the vehicle side are to be utilized so as to recognize movements more quickly and securely.

(44) FIG. 8 shows a further possibility if the camera is fixed in a holding device, in this case a mirror.

(45) In accordance with an exemplary embodiment of the present invention, it can be provided that the camera is moved or respectively the freezing of the image processing and display process in the mirror replacement system is recognized by way of the mirror adjustment.

(46) FIG. 9 shows a schematic representation of a flow chart of a method for checking the playback of a video sequence of a mirror replacement camera according to another exemplary embodiment of the present invention.

(47) FIG. 9 shows a sequence chart for recognizing an error in the image by adjusting the camera.

(48) FIG. 9 shows the result of the method. Since the camera is generally, in all probability, recording the ego vehicle or at least parts of the ego vehicle, a difference can also be clearly recognized here, for instance in the event that a turn signal light has been operated. Marking points could also be optionally affixed to the vehicle.

(49) FIG. 10 shows a schematic representation for checking the playback of a video sequence of a mirror replacement camera for a motor vehicle according to another exemplary embodiment of the present invention.

(50) The apparatus 100 for checking the playback of a video sequence of a mirror replacement camera for a motor vehicle comprises a camera device 10, an image data processing device 20, an output device 30, and a diagnosis device 40.

(51) The movable camera device 10 is designed to record an image and to provide image data.

(52) The image data processing device 20 is coupled to the camera device 10 and designed to produce the video sequence on the basis of the captured image data.

(53) The output device 30 is designed to display the produced video sequence.

(54) The diagnosis device 40 is designed to move the camera device 10 and/or to generate an image movement when producing the video sequence so as to check the displayed video sequence.

(55) FIG. 11 shows a schematic representation of a flow chart of a method for checking the playback of a video sequence of a mirror replacement camera for a motor vehicle according to another exemplary embodiment of the present invention.

(56) The method for checking the playback of a video sequence of a mirror replacement camera for a motor vehicle comprises the following method steps:

(57) As a first method step, an image is captured S1 and image data is provided by means of a movable camera device 10.

(58) As a second method step, the video sequence is produced on the basis of the captured image data.

(59) As a third method step, the produced video sequence is displayed S3.

(60) As a fourth method step, the camera device 10 is moved S4 and/or an image movement is generated when producing the video sequence so as to check the displayed video sequence.

(61) Although the present invention has been described above on the basis of preferred exemplary embodiments, it is not restricted to these, but can be modified in many ways. In particular, the invention can be amended or modified in multiple ways, without deviating from the core of the invention.

(62) In addition, it is pointed out that comprising and having do not exclude any other elements or steps and a or one does not exclude a plurality.

(63) It is additionally pointed out that features or steps, which have been described with reference to one of the above exemplary embodiments, can also be used in combination with other features or steps of other exemplary embodiments described above. Reference numerals in the claims are not to be viewed as restrictions.