Surround camera system for autonomous driving

Abstract

Disclosed herein is a surround camera system for autonomous driving. The surround camera system includes: at least one forward surveillance camera configured to monitor a region in front of a vehicle; at least one backward surveillance camera configured to monitor a region behind the vehicle; one or more side surveillance cameras configured to monitor both sides alongside the vehicle; a monitoring means configured to display each of acquired images; and a video control unit configured to process the images acquired via the forward surveillance camera, the backward surveillance camera, and the side surveillance camera, to transmit the processed images to the monitoring means, to detect a target object from the acquired images, analyze the target object, and issue a warning to a driver, and to control the electronic control unit of the vehicle; wherein a 360-degree omnidirectional region around the vehicle is monitored.

Claims

1. A surround camera system for autonomous driving, comprising: at least one forward surveillance camera configured to monitor a region in front of a vehicle; at least one backward surveillance camera configured to monitor a region behind the vehicle; one or more side surveillance cameras configured to monitor both sides alongside the vehicle; a monitoring means configured to display each of images acquired via the forward surveillance camera, the backward surveillance camera, and the side surveillance camera, to display a combined image obtained by combining two or more of the above images, or to display a dual image including at least any one of the above images and the combined image; a video control unit (VCU) configured to process the images acquired via the forward surveillance camera, the backward surveillance camera, and the side surveillance camera, to transmit the processed images to the monitoring means, to detect a target object obstructing traveling of the vehicle from the acquired images, analyze the target object, and issue a warning to a driver, and to control an electronic control unit (ECU) of the vehicle; a warning means connected to the video control unit and configured to call the driver's attention when it is determined that an emergency situation has occurred through detection of the target object; a global positioning system (GPS) connected to the video control unit; a manual switching button configured to manually switch an image from each of the images acquired via the forward surveillance camera, the backward surveillance camera, and the side surveillance camera, the combined image obtained by combining two or more of the above images, or the dual image including at least any one of the above images and the combined image to a desired image; and a warning light configured to indicate the emergency situation, wherein the forward surveillance camera comprises a visible light camera, a thermal observation device (TOD), and a front wide-angle camera; wherein the backward surveillance camera comprises: a first rear-view camera configured to monitor a right side of a region behind the vehicle, a second rear-view camera vehicle configured to monitor a left side of the region behind the vehicle, and a rear wide-angle camera configured to monitor the overall region behind the vehicle, including the left and right sides of the region behind the vehicle; wherein the forward surveillance camera, the backward surveillance camera, and the side surveillance cameras are modularized into a single component and then mounted on a roof panel of the vehicle; wherein the side surveillance cameras are disposed on left and right sides of the single component; wherein a 360-degree omnidirectional region around the vehicle is monitored via the forward surveillance camera, the backward surveillance camera, and the side surveillance cameras; wherein an image acquired via the TOD is displayed in an environment in which a recognition rate for the target object based on an acquired visual light image is low, and the target object is detected using a fact that a difference between an average temperature value on a road in a safe zone and a temperature value of the target object is high when the target object enters the safe zone; wherein the video control unit detects a distance between the target object and the vehicle, a traveling direction and traveling speed of the target object, a traveling direction and traveling speed of the vehicle, and a type of traffic signal light, and provides a warning, vehicle control, and guidance regarding a preset state; wherein the combined image comprises a top view image obtained by processing the images acquired via the forward surveillance camera, the backward surveillance camera, and the side surveillance cameras; wherein each of the front wide-angle camera and the rear wide-angle camera is a fisheye lens camera; wherein each of the first rear-view camera, the second rear-view camera, and the side surveillance cameras is a camera using a complementary metal oxide semiconductor (CMOS) sensor; wherein an angle of view of the TOD ranges from 20 to 40 degrees, an angle of view of the visible light camera ranges from 50 to 70 degrees, an angle of view of each of the front wide-angle camera and the rear wide-angle camera ranges from 110 to 130 degrees, an angle of view of each of the first rear-view camera and the second rear-view camera ranges from 75 to 85 degrees, and an angle of view of the side surveillance camera ranges from 80 to 100 degrees; and wherein, during a night or under adverse weather conditions, both images acquired via the visible light camera and the TOD are provided.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) The above and other objects, features and advantages of the present invention will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings, in which:

(2) FIG. 1 is a view conceptually showing the overall configuration of a surround camera system for autonomous driving according to the present invention;

(3) FIG. 2 is a view showing an embodiment of a state in which a forward surveillance camera, a backward surveillance camera, and side surveillance cameras used for the surround camera system for autonomous driving according to the present invention have been mounted;

(4) FIG. 3 is a view showing the angles of view and recognition distances of the cameras of the surround camera system for autonomous driving according to the present invention; and

(5) FIG. 4 is a view showing a case where a vehicle equipped with the surround camera system for autonomous driving according to the present invention has been applied to traveling on a road.

DETAILED DESCRIPTION

(6) The present invention may be subject to various alterations, and may have various embodiments. Specific embodiments will be illustrated in the accompanying drawings, and will be described in detail.

(7) However, this is not intended to limit the present invention to the specific embodiments, and it should be understood that the present invention encompasses all alterations, equivalents and substitutions falling within the spirit and technical range of the present invention. In the descriptions of the accompanying drawings, like reference symbols are used to designate like components.

(8) Although the terms “first,” “second,” etc. may be used to describe various components, the components should not be limited by these terms.

(9) These terms are each used only to distinguish one component from other components. For example, a first component may be termed a second component without departing from the scope of the present invention, and, in a similar way, a second component may be termed a first component. As used herein, the term “and/or” includes any or all combinations of one or more of associated listed items.

(10) It should be understood that when one component is described being “connected to” or “coupled to” another component, it may be connected or coupled directly to the other component or may be connected or coupled to the other component with a third component intervening therebetween. In contrast, it should be understood that when one component is described as being “connected directly to” or “coupled directly to” another component, it may be connected or coupled to the other component without a third component intervening therebetween.

(11) The terms used herein are used to illustrate a specific embodiment, and are not intended to limit the present invention. A singular description includes a plural description unless otherwise defined. In the present specification and the attached claims, it should be appreciated that the terms “include,” “comprise,” “have” and their derivatives are intended to designate the presence of features, numbers, steps, operations, components, parts or combinations thereof described in the specification or the attached claims, but are not intended to exclude the possibility of presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof in advance.

(12) Unless otherwise defined, all the terms used herein, including technical or scientific terms, have the same meanings as understood by those having ordinary knowledge in the technical field to which the present invention pertains. The terms commonly used and defined in dictionaries should be interpreted as having meanings identical to those specified in the context of related technology. Unless definitely defined therein, the terms should not be interpreted as having ideal or excessively formative meanings.

(13) Embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

(14) FIG. 1 is a view conceptually showing the overall configuration of a surround camera system for autonomous driving according to the present invention, FIG. 2 is a view showing an embodiment of a state in which a forward surveillance camera, a backward surveillance camera, and side surveillance cameras used for the surround camera system for autonomous driving according to the present invention have been mounted, FIG. 3 is a view showing the angles of view and recognition distances of the cameras of the surround camera system for autonomous driving according to the present invention, and FIG. 4 is a view showing a case where a vehicle equipped with the surround camera system for autonomous driving according to the present invention has been applied to traveling on a road.

(15) The surround camera system for autonomous driving according to the present invention will be described below with reference to FIGS. 1 to 4.

(16) The present invention is directed to a surround camera system for autonomous driving which can secure the forward view of a vehicle driver regardless of weather and climate conditions, which can guide a vehicle driver through safe driving, and which can be also applied to autonomous driving.

(17) Referring to FIG. 1, the surround camera system for autonomous driving according to the present invention is configured to include a forward surveillance camera 100, a backward surveillance camera 200, side surveillance cameras 300, a monitoring means 400, and a video control unit 500.

(18) The forward surveillance camera 100 includes at least one camera configured to monitor a region in front of a vehicle 10. In the present specification, there is proposed the forward surveillance camera 100 configured to include a visible light camera 120, a thermal observation device 110, and a front wide-angle camera 130.

(19) The visible light camera 120 may be a camera using a visible light sensor. Since this corresponds to well-known technology, a detailed description thereof is omitted.

(20) The thermal observation device 110 may be a camera using a thermal observation device (TOD) sensor. Since this corresponds to well-known technology, a detailed description thereof is omitted.

(21) The front wide-angle camera 130 is configured to monitor an overall region in front of the vehicle 10. It is preferable to use a fisheye lens camera as the front wide-angle camera 130.

(22) The backward surveillance camera 200 may include at least one camera configured to monitor a region behind the vehicle 10. In the present specification, the backward surveillance camera 200 including a first rear-view camera 210, second rear-view camera 220, and a rear wide-angle camera 230 is illustrated as an embodiment.

(23) Each of the first rear-view camera 210 and the second rear-view camera 220 may be a camera using a complementary metal oxide semiconductor (CMOS) sensor. It is preferable to use a fisheye lens camera as the rear wide-angle camera 230 configured to monitor an overall backward region.

(24) The side surveillance cameras 300 include one or more cameras configured to monitor both sides (left and right sides) of the vehicle 10. The side surveillance cameras 300 disposed on the left and right sides of the vehicle 10 are proposed as an embodiment.

(25) Each of the side surveillance cameras 300 may be a camera using a complementary metal oxide semiconductor (CMOS) sensor.

(26) The monitoring means 400 is installed inside the vehicle 10, and can be viewed by a driver. The monitoring means 400 displays each of the images acquired via the forward surveillance camera 100, the backward surveillance camera 200, and the side surveillance cameras 300, displays an image acquired by combining two or more of the above images, or displays a dual image including at least any one of the above images and the combined image.

(27) In other words, the monitoring means 400 displays each of the images acquired from the respective cameras, displays an image acquired by combining and converting two or more of the above images, or displays at least any one of the images acquired via the respective cameras and the converted image as a dual image. In this case, the term “dual” refers to two or more screens.

(28) Furthermore, the images acquired via the visible light camera 120 and the thermal observation device 110 are analyzed via the video control unit 500 under a climate condition, such as dark night, rain, snow, fog, yellow dust, backlight, or the like, and are provided to a driver as screens capable of improving a recognition rate for a target object.

(29) In other words, during a dark night, a recognition rate for a target object can be increased using an image acquired via the thermal observation device 110. An image acquired via the thermal observation device 110, rather than an image acquired via the visible light camera 120, is provided.

(30) However, during a night, a recognition rate for a target object can be increased by an image acquired via a thermal observation device 110, and the color of a signal light cannot be identified via the image. It is preferable to provide a dual image such that a recognition rate for a target object can be increased and also a signal light can be accurately identified via an image acquired via the thermal observation device 110 and an image acquired via the visible light camera 120.

(31) The video control unit (VCU) 500 processes images acquired via the forward surveillance camera 100, the backward surveillance camera 200, and the side surveillance cameras 300, transmits the processed images to the monitoring means 400, detects a target object obstructing the traveling of the vehicle 10 from the acquired images, analyzes the target object, issues a warning to a driver, and controls the electronic control unit (ECU) 510 of the vehicle 10.

(32) Referring to FIG. 2, there is shown a state in which the cameras according to the present invention have been mounted on the vehicle 10. The cameras may be modularized into a single component and then mounted on the vehicle 10, or may be individually mounted on the vehicle 10. The arrangement and mounting of the cameras may vary depending on the state and environment of the vehicle 10 in various manners.

(33) However, the present invention is configured to monitor a 360-degree omnidirectional region around the vehicle 10 via the forward surveillance camera 100, the backward surveillance camera 200 and the side surveillance cameras 300, and thus it is preferred that the front wide-angle camera 130 is installed on the upper portion of the center of the windshield of the vehicle 10 in order to set a reference point for images.

(34) Referring to FIG. 3, the angles of view and recognition distances of the cameras according to the present invention can be seen. It is preferred that the angle of view of the thermal observation device 110 ranges from 20 to 40 degrees, the angle of view of the visible light camera 120 ranges from 50 to 70 degrees, the angle of view of each of the front wide-angle camera 130 and the rear wide-angle camera 230 ranges from 110 to 130 degrees, the angle of view of each of the first rear-view camera 210 and the second rear-view camera 220 ranges from 75 to 85 degrees, and the angle of view of the side surveillance camera 300 ranges from 80 to 100 degrees.

(35) The recognition distances of the respective cameras are shown in the drawings. The angles of view of the respective cameras are configured as described above, and thus the cameras can monitor a 360-degree omnidirectional region around the vehicle 10.

(36) In this case, it is preferable to provide the combined image in the form of a top view image obtained by converting images acquired via the forward surveillance camera 100, the backward surveillance camera 200, and the side surveillance cameras 300.

(37) Referring to FIG. 4, an image of zone “A,” i.e., a space including side lanes on the left and right sides of the traveling lane of the vehicle 10 and areas on the front and back sides of the vehicle 10, is provided in the form of a top view image. A driver can easily recognize a vehicle, approaching his or her vehicle 10, via the monitoring means 400 by using the top view image.

(38) Furthermore, when a target object, such as another traveling vehicle located within a recognition distance, a joining traveling vehicle, or the like, appears in an image acquired via each of the cameras, the distance between the target object and the vehicle 10, the traveling direction and traveling speed of the target object, the traveling direction and traveling speed of the vehicle 10, and the type of traffic signal light are detected and a warning, vehicle control, and guidance regarding a preset state are provided, via the video control unit 500 connected to the GPS 530.

(39) Furthermore, a warning means 520, such as a monitor popup, a flicker, a speaker, or the like, is connected to the video control unit 500. A warning is provided to a driver in such a way that a target object is detected from the images acquired via the cameras and a warning is issued to call the driver's attention when an emergency situation occurs.

(40) As an example, the video control unit 500 measures a vehicle traveling speed via a GPS 530, and sets a safe zone (a safe distance) for the vehicle traveling speed. In other words, the safe distance may be set to 30 m for a speed of 30 km/h, the safe distance may be set to 60 m for a speed of 60 km/h, and the safe distance may be set to 100 m for a speed of 100 km/h.

(41) Accordingly, the safe distance previously set for the vehicle traveling speed is recognized, and a vehicle traveling speed at the moment a target object is detected is measured via the GPS 530 when the target object is detected within the safe distance. When the measured vehicle traveling speed exceeds a speed based on the safe distance, a warning is issued to a driver via at least any one of a monitor and a speaker depending on the vehicle traveling speed. In the case of the monitor, a warning may be issued to a driver by using one or more methods, such as a popup window, a flicker, a flicker of a red screen, and/or the like. In the case of the speaker, a warning may be issued to a driver by using one or more of well-known various methods, such as a warning sound, a speed reduction announcement, and/or the like.

(42) Furthermore, in addition to the control regarding the safe zone (the safe distance), when the yellow or red signal of traffic signal lights is detected, the distance to the vehicle 10 may be determined, a vehicle traveling speed may be measured, and a warning may be issued using a method identical to the above-described method.

(43) Meanwhile, when the vehicle traveling speed is not reduced after a warning has been issued to the driver, the vehicle traveling speed may be reduced to a preset state in a stepwise manner by controlling the electronic control unit 510 mounted on the vehicle 10. Since the electronic control unit 510 is connected to the video control unit 500, the electronic control unit 510 may be controlled via the video control unit 500. During the stepwise reduction of the speed, a driver is prompted to recognize a surrounding situation and take measures by herself or himself by primarily calling a driver's attention via automatic braking.

(44) Furthermore, when measures are not taken by a user after automatic braking has been performed one or more times, automatic braking is successively performed for each predetermined distance. Control is performed such that the vehicle 10 can be completely stopped in front of a target object before the occurrence of a bump.

(45) As described above, a recognition rate for a target object is increased and countermeasures against an emergency situation are automatically taken, and thus a vehicle is guided through safe autonomous driving and the safety of a driver and a pedestrian can be improved.

(46) Meanwhile, when a target object is detected within a safe zone during the traveling of the vehicle in a dark night, the target object is detected by analyzing a difference in temperature via the thermal observation device 110.

(47) In other words, a temperature within the safe zone is chiefly the temperature of the traveling road of the vehicle, and a difference in temperature is small across the traveling road. When another vehicle, a human, an animal, or the like enters within a safe distance, a difference in temperature between the entering object and the traveling road increases. Accordingly, a target object can be immediately recognized via the difference in temperature, and thus a recognition rate for a target object can be improved.

(48) Furthermore, there may be further included a communication means, such as a Bluetooth means or the like, connected to the video control unit 500. The surround camera system may be configured to operate in cooperation with a smartphone or navigation system via the communication means. The cooperative operation with the smartphone and the navigation including mirroring, etc. corresponds to well-known technology, and thus a detailed description thereof is omitted.

(49) Meanwhile, referring to FIG. 1, a manual switching button 540 connected to the video control unit 500 may be provided, and may be used to manually switch a current image from each of the images acquired via the forward surveillance camera, the backward surveillance camera, and the side surveillance camera, the combined image obtained by combining two or more of the above images, or the dual image including at least any one of the above images and the combined image to a desired image. Although an embodiment is conceptually shown in the drawing, the location, shape, size, type, etc. of the manual switching button 540 may be subject to various modifications within the scope of well-known technology.

(50) Furthermore, a warning light 550 configured to indicate an emergency situation is further provided, and thus separate equipment, such as a warning triangle or the like, is not required in the case of an accident or emergency situation and information about the accident or emergency situation can be rapidly transferred to surrounding vehicles.

(51) According to the above-described present invention, an image of a 360-degree omnidirectional region around a vehicle can be acquired via a plurality of cameras configured to monitor front, rear and side regions in front of, behind and alongside the vehicle, a target object obstructing the traveling of the vehicle can be identified through the analysis of the acquired image, a warning can be issued to a driver, and the vehicle can be safely guided by controlling the electronic control unit of the vehicle when countermeasures are not taken after the issuance of the warning.

(52) Furthermore, images acquired via the visible light camera and the thermal observation device are provided, so that a highly recognizable image and information about forward safety can be provided to a driver regardless of weather and climate conditions (such as backlight, fog, snow, rain, yellow dust, night, etc.).

(53) Furthermore, the distance between a target object and a vehicle, the traveling direction and traveling speed of the target object, the traveling direction and traveling speed of the vehicle, and the type of traffic signal light can be detected and a warning, vehicle control, and guidance regarding a preset state can be provided, so that the technical level of an advanced driver assistance system (ADAS) at its early stage can be improved.

(54) Furthermore, automatic image provision can be performed depending on a climate environment, and a user can easily switch a current image to a desired image via a manual switching button.

(55) Furthermore, the warning light is provided, so that separate equipment, such as a warning triangle or the like, is not required in the case of an accident or emergency situation and information about the accident or emergency situation can be rapidly transferred to surrounding vehicles.

(56) Moreover, an image of a 360-degree omnidirectional region around a vehicle is provided in the form of a top view image, so that a situation around the vehicle can be rapidly and easily detected.

(57) Although the specific embodiments of the present invention have been disclosed for illustrative purposes, those skilled in the art will appreciate that various modifications, additions and substitutions are possible without departing from the scope and spirit of the invention as disclosed in the accompanying claims.