CAMERA MONITROING SYSTEM FOR MOTOR VEHICLES
20200401818 · 2020-12-24
Inventors
- Lluís Gibert Castroverde (Viladecavalls, ES)
- Natalia Canosa Perez (Viladecavalls, ES)
- Noelia Rodriguez Ibañez (Viladecavalls, ES)
- Brenda Meza García (Viladecavalls, ES)
- Jordi Tenas Martínez (Viladecavalls, ES)
- Xavier Biosca Yuste (Viladecavalls, ES)
- Daniel Guerra Fagundez (Viladecavalls, ES)
Cpc classification
B60R1/12
PERFORMING OPERATIONS; TRANSPORTING
B60W2554/4048
PERFORMING OPERATIONS; TRANSPORTING
B60R2001/1253
PERFORMING OPERATIONS; TRANSPORTING
B60R1/00
PERFORMING OPERATIONS; TRANSPORTING
B60R2001/1284
PERFORMING OPERATIONS; TRANSPORTING
G06V20/56
PHYSICS
B60R2300/207
PERFORMING OPERATIONS; TRANSPORTING
B60K35/00
PERFORMING OPERATIONS; TRANSPORTING
B60R2300/8046
PERFORMING OPERATIONS; TRANSPORTING
International classification
Abstract
A camera monitoring system is adapted for use in vehicles, and includes an image capturing means, a control unit, and at least one display device. The image capturing means is configured to capture an image from an external environment, and is associated with an exterior rear-view mirror of the vehicle. The unit is connected to the capturing means, and is configured to select an image region from the captured image. The image region is smaller than the captured image and is movable within the captured image. The device may be a touch screen, and the system may include a control surface in the touch screen configured to move via touches performed by a driver. The image region is displayed in the touch screen, or in another, additional, screen of the device. The system may further include a gesture detector for the detection of driver gestures associated with the image region.
Claims
1. A camera monitoring system for motor vehicles, comprising: image capturing means being associated with at least an exterior rear-view mirror of a vehicle, the exterior rear-view mirror being a mounting assembly located at an exterior part of the vehicle, the image capturing means being located at the mounting assembly and configured to capture an image from an exterior field of view of the vehicle, wherein the field of view extends at least sideward and rearward outside the vehicle and encompasses a portion of the exterior part of the vehicle; an electronic control unit, ECU connected to the image capturing means and configured to select at least an image region from the image captured by the image capturing means, the image region being smaller than the captured image and movable within the captured image; and at least one display device including at least a first screen located inside the vehicle and connected to the ECU; wherein a control surface in the first screen is multi-touch sensitive, and/or is a gesture detector, configured both to move the image region within the captured image, and wherein the image region being displayed by the, at least one, display device.
2. The camera monitoring system according to claim 1, wherein the first screen of the display device is a touch screen and the display device comprises two different portions, a first portion in which a touch- and-drag of the displayed image is disabled, and a second portion in which the touch-and-drag provided by the touch screen is activated by the ECU.
3. The camera monitoring system according to claim 2, wherein further comprising a frame covering partially the touch screen to split the first display device into at least the first portion and the second portion.
4. The camera monitoring system according to claim 3, wherein the frame is partially tinted and comprises at least one non-tinted portion corresponding to the second portion of the first display device.
5. The camera monitoring system according to claim 4, wherein the frame comprises two non-tinted portions, a first non-tinted portion corresponding to the second portion displaying the image region adjustable in size or location on the touch screen by a user's touch and a second non-tinted portion corresponding to the second portion displaying control items to control parameters through the control surface of the touch screen.
6. The camera monitoring system according to claim 3, wherein the parameters to be controlled through the control surface of the touch screen are selected from brightness, contrast and color parameters of the images displayed by the display device.
7. The camera monitoring system according to claim 5, wherein the parameters to be controlled through the control surface of the touch screen are selected from brightness, contrast and color parameters of the images displayed by the display device.
8. The camera monitoring system according to claim 3, wherein the parameters to be controlled through the control surface of the touch screen are selected from height of the field of view of the image capturing means and width of the field of view of the image capturing means.
9. The camera monitoring system according to claim 7, wherein the parameters to be controlled through the control surface of the touch screen are selected from height of the field of view of the image capturing means and width of the field of view of the image capturing means.
10. The camera monitoring system according to claim 3, wherein the control items are selected from buttons drawn on the frame, digital buttons and digital menus.
11. The camera monitoring system according to claim 9, wherein the control items are selected from buttons drawn on the frame, digital buttons and digital menus.
12. The camera monitoring system according to claim 2, wherein the first portion and the second portion of the first display device are the same touch screen.
13. The camera monitoring system according to claim 11, wherein the first portion and the second portion of the first display device are the same touch screen.
14. The camera monitoring system according to claim 2, wherein the display device further comprises a second screen which is not touch-sensitive and the first portion of the display device is the second screen and the second portion of the display device is the touch screen.
15. The camera monitoring system according to claim 11, wherein the display device further comprises a second screen which is not touch-sensitive, and the first portion of the display device is the second screen and the second portion of the display device is the touch screen.
16. The camera monitoring system according to claim 1, further comprising an additional camera inside the vehicle configured to capture driver's gestures and the gesture detector is implemented by the ECU using an image classifier.
17. The camera monitoring system according to claim 1, wherein the gesture detector is implemented by a touch-sensitive screen with increased capacity sensibility of the display device.
18. The camera monitoring system according to claim 1, wherein the gesture detector is implemented by an electronic device based on electrical near-field gesture controllers.
19. The camera monitoring system according to claim 1, wherein the image capturing means are fixedly located outside the vehicle.
20. A motor vehicle comprising: an intelligent rear-view monitoring system, IRMS, with an image capturing means associated with an interior rear-view mirror, the IRMS including; a first camera monitoring system according to claim 1 with image capturing means located on the left side of the vehicle and associated with an exterior rear-view mirror of the left side, a second camera monitoring system according to claim 1 with image capturing means located on the right side of the vehicle and associated with an exterior rear-view mirror of the right side, and wherein a single ECU connected to all the image capturing means is configured to control the first camera monitoring system and the second camera monitoring system, and is further configured to control the IRMS.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0023] For the purpose of aiding the understanding of the characteristics of the invention, according to a preferred practical embodiment thereof and in order to complement this description, the following Figures are attached as an integral part thereof, having an illustrative and non-limiting character:
[0024]
[0025]
[0026]
[0027]
[0028]
[0029]
[0030]
[0031]
[0032]
[0033]
[0034]
[0035]
[0036]
[0037]
DETAILED DESCRIPTION
[0038] The matters defined in this detailed description are provided to assist in a comprehensive understanding of the invention. Accordingly, those of ordinary skill in the art will recognize that variation changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, description of well-known functions and elements are omitted for clarity and conciseness.
[0039] Of course, the embodiments of the invention can be implemented in a variety of architectural platforms, operating and server systems, devices, systems, or applications. Any particular architectural layout or implementation presented herein is provided for purposes of illustration and comprehension only and is not intended to limit aspects of the invention.
[0040]
[0041] For example,
[0042] The CMS further comprises an electronic control unit or ECU connected to the image capturing means, configured to select an image region from the image captured by the image capturing means. The image region is smaller than the captured image; i.e., is obtained by cropping the image from the image capturing means.
[0043] The CMS further comprises at least one (first) display device (100, 101, 102) located inside the motor vehicle (10), for example as shown in
[0044] The control surface can be implemented in a touch screen (200) of the (first) display device (100, 101, 102) preferably working as a multi-touch-sensitive display configured to receive a user's touch so as to move the image region within the captured image.
[0045] Preferably, the touch screen (200) is configured to detect different places touched simultaneously, in particular by a finger. The multi-touch-sensitive display allows tracking of images and touch carried out not only by one finger, but also complex actions, requiring operation with two or more fingers, can be performed. With particular advantage, the multi-touch-sensitive display may be further provided with a magnification of image regions by using two fingers and a change in the spacing of the fingers during contact is adjustable. In particular, together with the panning, a simple adjustment of the desired composition including the zoom level (zoom in/out) can be given by simple movements of the fingers. Therefore, the CMS comprises an associated touch screen controller configured to detect also movements and/or touches of one or more fingers by increasing the sensitive capacity of the touch screen (200).
[0046] In the case of exterior rear-view mirrors (110, 111) and/or wing mirrors, as shown in
[0047] In another embodiment, the touch screen (200) can be additionally arranged for an interior rear-view mirror (112) in an interior part of the motor vehicle (10). Typically, the interior rear-view mirror (112) and the exterior rear-view mirrors (110, 111) are connected through the ECU of the vehicle (10).
[0048] The multi-touch-sensitive screen (200) of the display device (100, 101, 102) can be configured to be an only-touch screen. That is, instead of having a touch and drag screen, the CMS provides the touch screen (200) with two different portions as the ECU (500) distinguishes: a first portion of the touch screen (200) in which the user/driver is not enabled to perform touch and drag of the displayed image, and a second portion in which the user/driver can perform normally the touch and drag operation provided by the multi-touch-sensitive screen (200).
[0049] According to this example, the user can perform only touch in the first portion or touch & drag in the second portion. The technical advantage of doing so is that the first portion does not get dirty. In this example, the light (image) received by the second portion comes from the same image of the display as the first portion. Preferably, the first portion is larger than the second portion. Preferably, the area of the second portion is approximately 30% of that of the first portion. Preferably, the length of the second portion is 20% of the length of the first portion, the length being defined along an X (horizontal) axis.
[0050] Therefore, instead of having a screen on each side (on the inside of the door) of the vehicle (10) and the screen being totally touch and drag for its whole surface, according to a possible embodiment, a single touch screen (200) on each side of the vehicle (10) is connected to the ECU (500), which distinguishes between a first portion (2001) and a second portion (2002) of the touch screen (200), as shown in
[0051] In a preferred embodiment, in order to implement the aforementioned two portions (2001, 2002) in the first display device (100, 101, 102), the CMS further comprises a frame (300) covering partially the first display device (100, 101, 102), as shown in
[0052] This frame has the same dimensions as the touch screen (200) or significantly larger to cover at least the entire touch screen (200). The frame is a cover, preferably made of plastic or glass, which also protects the display device (100, 101, 102) from impacts and damages, since the touch screen (200) is relatively fragile. The frame is partially tinted. Preferably, the tinting is black. Said frame does not allow all the light emitted by the touch screen (200) can pass. Therefore, the driver does not see the light emitted by the display where the frame is tinted, i.e., silkscreen. The frame is placed, for example, on top of the touch screen (200), and so what the user sees is the frame, since it is placed between the screen and the user.
[0053] As described before, the frame can be partially tinted; more particularly, comprising a (first) section that is not tinted, while the rest of the frame is stained. The transparent or non-tinted portion of the frame, from which the light of the display comes out and becomes visible by the user, can present different forms. Therefore, only the section of the frame (image) that is not tinted is shown/perceived to/by the user, but not all the image emitted by the display. According to a non-limiting example, as shown in
[0054] The ECU (500) deactivates the first portion of the touch screen (200) according to the specific geometry of the (first) non-tinted section of the frame.
[0055] According to another possible embodiment, the first display device (100, 101, 102) has an additional, second screen (400) in addition to the multi-touch-sensitive screen (200). Having two screens, as shown in
[0056] In a further possible embodiment, the CMS can take advantage of the second portion (2002) (implemented in a single touch screen (200) or in an additional screen as described before) to indicate a Blind Spot Detection or BSD, i.e., if an object is detected in the blind spot, the CMS turns on an indicator (triangle, exclamation, etc.) in said second portion (2002) of the first display device (100, 101, 102).
[0057] The CMS can be additionally provided with at least one (second) display device (another touch screen) to display parameters of the first display device (100, 101, 102), i.e., operation parameters of the touch screen (200). The first display device (100, 101,102) itself can also display the parameters to be adjusted. More particularly, the touch screen (200) allows the user by his/her touch to set brightness and/or contrast and/or color parameters of the first display device (100, 101, 102). Therefore, parameters are adjustable directly on the touch screen of the display device with respect to the optical impression of the image display parameters shown on the display. The driver can make the image lighter or darker, to choose contrast and other common parameters, in particular color-related parameter set of the images displayed in the touch screen (200). In a possible embodiment, the CMS is designed for inserting or superimposing a slider bar for the presentation parameters on the display device.
[0058] Additionally, at least one operating parameter of the first display device (100, 101, 102) can be used by the CMS to output an item of information and/or a warning, in particular to be used by a driver's assistance system (e.g., a lane change assistance system and/or a blind spot assistance system and/or parking assistance and/or a reversing assistance system) provided in the vehicle (10). The output of this information and/or a warning is adjustable depending upon an input on the touch screen (200).
[0059] Additionally, at least one item of additional information can be superimposed into the image displayed on the touch screen (200) and can be moved, by interaction at the display location of the additional information in the touch screen (200). In another embodiment, context-sensitive menus of items which can be activated by the touch screen (200) and displayed on the display device (100, 101, 102).
[0060] The ECU (500) or control unit can provide some digital buttons to be overlaid on the image shown in a second portion (2002) of the touch screen (200). For example, after selecting a specific adjustment possibility which can be done through a menu and/or a depicted control, thus a slider is displayed on the control surface of the touch screen (200), which can be gripped, for example by contact at the position of the slider and manipulated to adjust the display parameters. Of course, other possibilities are conceivable to make the presentation parameter set, for example with + and/or labeled on the display area on-screen controls that trigger corresponding to an increase or decrease of the presentation parameter if a touch is sensed (zoom function). Particularly advantageously, it is also when the touch screen (200) can also be used to set further parameters. The control unit can be configured to form a touch screen menu on the screen (200). Thus, when the menu is selected by a touch, a menu can be displayed, which allows the setting of parameters of the winglet. The driver can touch with a finger (or with a pen) the image portion on one of the labelled symbols and cause a setting menu open different options (or additional information) in the image portion. Therefore, overlaid symbols are then placed with a menu having multiple menu items related to display parameters; in particular, for contrast, brightness and color parameters. It is also conceivable that by tapping the touch screen shown outside controls/menu items return to the default display. Additionally, touching the additional information can result in a corresponding menu with settings concerning the lane change assistant, i.e., in particular operating parameters of the lane change assistant.
[0061] Another option is to draw (or taint) buttons (310) on the frame (300), as depicted in
[0062] According to an example, the frame and the screen (200) are together and in contact. According to another example, there is a gap between the frame and the screen (200) and preferably the gap is between 0.5 mm-4 mm. According to a further example, a joystick including the typical mouse ball (not optical) for computers or physical push buttons can be used. The joystick and/or the push buttons can be located anywhere the driver has access when driving; for example, on the door, on the dashboard (central console), on the steering wheel, etc. The push buttons can have a cross-shaped distribution, in order to move up/down/left/right. The joystick can have a cross or circular movement which allows the image to be moved according to the movement of the joystick. Switches or auxiliary pushbuttons can allow the user to choose between the different options (e.g., left or right CMS). A trackball can also be used, which allows greater precision in adjusting the position of the image since the trackball detects small movements of the ball. Also, a touchpad can be used, as the press of a button can be simulated by pressing the touchpad, so that the user can move the finger across the length and width of the pad to determine the movement of the image. Other types of elements sensitive to human touch such as plastic material or smart textiles can be used. All these above-mentioned elements can be placed in different elements of the car, (without being limiting): door, driver's side panel, center console, central tower (central armrest), steering wheel, etc.
[0063]
[0064] More particularly,
[0065] In a possible embodiment, eachright, leftwinglet of the CMS can comprise:
a) a camera focused sensibly backwards, which is the so-calledright, leftCMS camera (1102, 1112);
b) a camera focused sensibly down, which is theright, lefttop-view camera (1101, 1111);
c) a camera focused sensibly forward, which is useful, for example, when the driver overtakes. Especially when he/she overtakes the vehicle in front on the right, there is no visibility of what is ahead (e.g. in the area adjacent in front of the vehicle); then, thanks to this front camera, the image of what is ahead can be displayed in the second portion (2002), preferably displayed by a second screen (400).
[0066]
[0067] Optionally the image region (211) displayed in the second portion of the left display device (101) may further comprise an additional image section (232), as shown in
[0068] Optionally, as shown for the left display device (101) and the right display device (100) in
[0069] According to a variation of the previous embodiment, in the case that the rear camera (1400) is left without an image to be displayed in the second portion of the display devices (100, 101), the image from the corresponding top view camera (1101, 1111) of the display devices (100, 101) can be taken. This is useful when the image region (210, 211), as being moved inside the captured image, can reach an edge (extreme) of the captured image. In this case, since the image region (210, 211) is shown on the first portion (2001), there is no image to be shown on the second portion (2002); however, when this happens, the ECU (500) detects it and selects the top view image to be shown on the second portion (2002). The ECU (500) can select: (i) the top view of the top-view camera of a single CMS device; or (ii) the top-view of the surrounding view, that is, after doing the stitching of the four top-view images from left, right, front and rear cameras. Optionally, instead of showing the top-view, the image from the area (1222) captured by the rear camera (1400), i.e., it is not all the rear image, but the portion of the adjacent/complementary rear image, can be displayed on the second portion (2002) or second screen (400).
[0070] In a possible embodiment, the display devices (100, 101, 102) can be controlled by the user, driver or co-driver, from a smartphone or tablet, e.g., via Bluetooth. The CMS can automatically load the settings previously input by the driver (i.e., user preferences of the device owner: default position, views . . . ) into his/her smartphone or tablet to the ECU (500).
[0071] In a further embodiment, the controller or ECU (500) of the CMS is configured also to receive and process vehicle driving and/or user information, and the image region (210, 211) is moved within the captured image depending on said received information. The vehicle driving information at least corresponds to a change in driving direction from a forward driving direction to a reverse driving direction or vice versa, an increase or decrease in driving speed with respect to a predefined driving speed value, a lane change, a change in steering angle, a change in pitch angle, a change in roll angle, and road monitoring information.
[0072] In an example, the ECU (500) is further configured to: [0073] move the image region (210, 211) corresponding to a downward vertical displacement such that the field of view, FOV, is vertically displaced in a downward direction, when the driving direction is changed from a forward driving direction to a reverse driving direction, and [0074] move the image region (210, 211) corresponding to an upward vertical displacement such that the field of view, FOV, is vertically displaced in an upward direction, when the driving direction is changed from a reverse driving direction to a forward driving direction.
[0075] In a further example, the ECU (500) is configured to determine a relative upward and downward movement of the exterior rear-view mirror (110, 111) from a change in the pitch angle value, and further configured to: [0076] move the image region (210, 211) corresponding to a downward vertical displacement such that the field of view, FOV, is vertically displaced in a downward direction, when the exterior rear-view mirror (110, 111) is downwardly moved, and [0077] move the image region (210, 211) corresponding to an upward vertical displacement such that the field of view, FOV, is vertically displaced in an upward direction, when the exterior rear-view mirror (110, 111) is upwardly moved.
[0078] In a further example, the ECU (500) is configured to receive a lateral displacement of the vehicle, for example, by a change in the steering angle value, and further configured to: [0079] move the image region (210, 211) corresponding to a rightward lateral displacement such that the field of view is laterally displaced in a leftward direction in the event of a left lane change, and [0080] move the image region (210, 211) corresponding to a leftward lateral displacement such that the field of view is laterally displaced in a rightward direction in the event of a right lane change.
[0081] In a further example, the ECU (500) is configured to move the image region (210, 211) such that the zoom of the displayed image increases when the driving speed is increased and such that it surpasses the predefined driving speed value, while the image region area is decreased when the vehicle speed falls below a predefined driving speed value. Thus, a smaller part of the captured image is selected as image region (210, 211), and then an enlarged image is displayed when over speed is detected by the ECU (500). In this way, the driver has a better view of the area of interest, which in this case, corresponds to a more detailed view of the farthest area from the vehicle.
[0082] Preferably, the ECU (500) is configured to receive an activation signal such a blinker, and further configured to move the image region (210, 211) upon receiving said activation signal.
[0083] In a further embodiment, the CMS can work with speed or other measurements (e.g., if the reverse gear is engaged in parking situations) obtained from the CAN, or another network (e.g., Ethernet, etc.) which the ECU (500) is communicated with. The same measured speed signal can be used to adjust the FOV of the entire system in the vehicle (10), shown in
[0084] The driver can adjust the FOV of the IRMS through the touch & drag, preferably provided by the second portion (2002) of the CMS display device (100, 101, 102). While traditional central rear-view mirrors are tilted because the driver is on the left side of the car and wants to see what is behind, the interior mirrors of the IRMS, which provides the driver with a display showing the image of the rear camera (1400), are centered. There is no option to turn the rear-view mirror in the IRMS, but the driver can perform a crop & pan of the rear camera through the touch & drag functionality in the second portion (2002). In crop & pan functionality of the IRMS, the rear camera (1400) captures an image, in which a controller-either a single ECU of the vehicle, any of the CMS's ECUs (500), or the IRMS's ECU-selects a portion of the image or image region that appears on the IRMS display.
[0085] According to another preferred embodiment, the CMS comprises, as alternative to the before described approach based on a control surface of the touch screen or in addition to it, a gesture detector so as, for example, to move the image region within the image captured by the exterior image capturing means by gestures of the driver's fingers or to allow the driver command the CMS to expand the FOV by a head movement. There are three implementation options for the gesture detector:
[0086] i) In a possible embodiment, gestures can be recognized in images of the user captured by an additional camera inside the vehicle (10), for example, near the user's display or in the interior mirror, and the ECU is configured to use an image classifier by to avoid false positives and false negatives in the detected gestures
[0087] ii) In another possible embodiment, no extra camera is needed, the gestures are performed on the display without touching it and they are detected by simply increasing the capacity sensibility of its screen.
[0088] iii) A further possible embodiment is based on electrical near-field (E-field) 3D gesture controllers that enable user gesture detection and motion tracking by using a single-chip device with no host processing needed for embedded applications. Thus, only by turning his/her head or waving one hand, the CMS detects that the driver wants an opening (greater) of his/her FOV, and consequently increases the FOV of the external camera. Therefore, this preferred embodiment provides multiple possible locations from which the driver can make the movements.
[0089] In another embodiment of the CMS using a gesture detector, the crop-and-pan functionality may be triggered by a driver's head movement. The driver's head movement may be tracked by a surveillance system, which is preferably within the vehicle, more preferably fixed (i) in front of the driver, or (ii) in the interior rear-view mirror, (iii) or near to the CMS display device. Preferably, the driver surveillance system may comprise a camera. The surveillance system may be capable of working out the position and distance of the driver's head. Thus, the image region (210, 211) within the captured image may be manually controlled (e.g., by screen sliding command o by the touch-and-drag functionality), but also by the driver's head moving forward when this movement is captured by the gesture detector.
[0090] Note that in this text, the term comprises and its derivations (such as comprising, etc.) should not be understood in an excluding sense, that is, these terms should not be interpreted as excluding the possibility that what is described and defined may include further elements, steps, etc.
[0091] While the present disclosure has been described with reference to an exemplary embodiment or embodiments, it will be understood by those skilled in the art that various changes may be made, and equivalents may be substituted for elements thereof without departing from the scope of the present disclosure. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the present disclosure without departing from the essential scope thereof. Therefore, it is intended that the present disclosure not be limited to the particular embodiment disclosed as the best mode contemplated for carrying out this present disclosure, but that the present disclosure will include all embodiments falling within the scope of the claims.