IN-VEHICLE SENSOR SYSTEM

Abstract

A system of the present disclosure includes a coarse observation sensor configured to observe a range around a vehicle, high-accuracy observation object identification means configured to identify a high-accuracy observation object that is an object detected by the coarse observation sensor in the observation range and is an object to be observed at a higher resolution, object presence area prediction means configured to predict a range of an object future presence area where the high-accuracy observation object may be present after the identification, a fine observation sensor configured to observe the range of the object future presence area at the higher resolution, and object information output means configured to output information on the high-accuracy observation object observed by the fine observation sensor.

Claims

1. An in-vehicle sensor system configured to observe a situation around a vehicle, the in-vehicle sensor system comprising: a first sensor configured to observe a predetermined range around the vehicle at a first resolution; high-accuracy observation object identification means configured to identify a high-accuracy observation object, the high-accuracy observation object being an object detected by the first sensor in the predetermined range and being an object to be observed at a second resolution, the second resolution being higher than the first resolution; object presence area prediction means configured to predict a range of an object future presence area, the object future presence area being an area where the high-accuracy observation object may be present after the identification; a second sensor configured to observe the range of the object future presence area at the second resolution; and object information output means configured to output information on the high-accuracy observation object observed by the second sensor.

2. The in-vehicle sensor system according to claim 1, wherein: the high-accuracy observation object identification means is configured to detect a position or range of a presence area of the high-accuracy observation object in the predetermined range observed by the first sensor; and the object presence area prediction means is configured to predict a position or range of the object future presence area seen from the vehicle, based on the position or range of the presence area of the high-accuracy observation object in the predetermined range.

3. The in-vehicle sensor system according to claim 2, wherein: the high-accuracy observation object identification means is further configured to detect a type of the high-accuracy observation object; and the object presence area prediction means is configured to predict the position or range of the object future presence area seen from the vehicle, based on the position or range of the presence area of the high-accuracy observation object in the predetermined range and, in addition, based on the detected type of the high-accuracy observation object.

4. The in-vehicle sensor system according to claim 2, the system further comprising vehicle motion state acquisition means configured to acquire a vehicle speed or moving distance, and/or a turning state value or turning angle, of the vehicle, wherein the object presence area prediction means is configured to predict the position or range of the object future presence area seen from the vehicle, based on the position or range of the presence area of the high-accuracy observation object in the predetermined range and, in addition, based on the vehicle speed or moving distance, and/or the turning state value or turning angle, of the vehicle.

5. The in-vehicle sensor system according to claim 3, wherein the object presence area prediction means is configured to predict the object future presence area that varies in size depending upon the type of the high-accuracy observation object.

6. The in-vehicle sensor system according to claim 2, wherein: the high-accuracy observation object identification means is further configured to detect a relative speed and/or a relative moving direction of the high-accuracy observation object seen from the vehicle; and the object presence area prediction means is configured to predict the position or range of the object future presence area seen from the vehicle, based on the position or range of the presence area of the high-accuracy observation object in the predetermined range and, in addition, based on the detected relative speed and/or detected relative moving direction of the high-accuracy observation object.

7. The in-vehicle sensor system according to claim 6, the system further comprising vehicle motion state acquisition means configured to acquire a turning state value or turning angle of the vehicle, wherein the object presence area prediction means is configured to predict the position or range of the object future presence area seen from the vehicle, based on the position or range of the presence area of the high-accuracy observation object in the predetermined range, based on the relative speed and/or relative moving direction of the high-accuracy observation object, and based on the turning state value or turning angle of the vehicle.

8. The in-vehicle sensor system according to claim 1, wherein the high-accuracy observation object identification means is configured to include detected-object threat level determination means to determine the high-accuracy observation object based on a threat level of an object, the detected-object threat level determination means being configured to determine the threat level of the object, the threat level representing a level of an impact of the object on traveling of the vehicle, the object being an object detected in the predetermined range observed by the first sensor.

9. The in-vehicle sensor system according to claim 8, wherein the high-accuracy observation object identification means is configured to select at least one object in descending order of the threat level as the high-accuracy observation object.

10. The in-vehicle sensor system according to claim 1, wherein the first and second sensors are sensors selected from a camera, a millimeter wave radar, and a rider.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Features, advantages, and technical and industrial significance of exemplary embodiments of the disclosure will be described below with reference to the accompanying drawings, in which like signs denote like elements, and wherein:

[0020] FIG. 1A is a schematic diagram of a vehicle to which an embodiment of an in-vehicle sensor system according to the present disclosure is applied;

[0021] FIG. 1B is a block diagram showing the configuration of the system in one embodiment of the in-vehicle sensor system according to the present disclosure;

[0022] FIG. 2A is a flowchart showing the operation of the system during the observation of the situation around a vehicle in the in-vehicle sensor system in the embodiment when the expected position of a high-accuracy observation object is predicted based on the type;

[0023] FIG. 2B is a flowchart showing the operation of the system during the observation of the situation around a vehicle in the in-vehicle sensor system in the embodiment when the expected position of a high-accuracy observation object is predicted based on the relative speed between the high-accuracy observation object and the vehicle;

[0024] FIG. 3A to FIG. 3D are plan views each showing the positional relationship between an object observed by the in-vehicle sensor system in the embodiment and the vehicle and showing the processing for predicting a future presence area, which is an expected position, based on the type of a high-accuracy observation object and for determining an observation range to be observed by a fine observation sensor; and

[0025] FIG. 4A to FIG. 4D are plan views each showing the positional relationship between an object observed by the in-vehicle sensor system in the embodiment and the vehicle and showing the processing for predicting a future presence area, which is an expected position, based on the relative speed between a high-accuracy observation object and the vehicle and for determining an observation range to be observed by the fine observation sensor.

DETAILED DESCRIPTION

Configuration of In-Vehicle Sensor System

[0026] With reference to FIG. 1A, an embodiment of an in-vehicle sensor system of the present disclosure will be described. A vehicle 10 such as an automobile includes a coarse observation sensor 14, a fine observation sensor 16, and an observation control device 12. The coarse observation sensor 14 observes the situation around the vehicle 10 at a first resolution. The fine observation sensor 16 observes the situation around the vehicle 10 at a second resolution that is higher than the first resolution. The observation control device 12 controls the operation of the coarse observation sensor 14 and the fine observation sensor 16. In addition, the observation control device 12 receives signals from the coarse observation sensor 14 and the fine observation sensor 16 and, from the received signals, detects and recognizes the presence or absence of objects (such as other vehicles, roadside buildings, walls, fences, guardrails, poles, parked vehicles, pedestrians (pedestrians, bicycles) road ends, road markings (white lines, yellow lines,) and traffic lights), their positions or presence ranges, speeds, moving directions, or types, and outputs the result. The coarse observation sensor 14 may typically be a camera that captures the situation around the vehicle or may be a sensor, such as a millimeter wave radar or a lidar, that magnetically or optically scans and observes the situation around the vehicle. More specifically, as will be understood from the description below, the purpose of the coarse observation sensor 14 is to extensively and quickly observe the whole range, which is to be observed around the vehicle, for detecting the presence of an object that is around the vehicle and that may affect the traveling of the vehicle. Therefore, as the coarse observation sensor 14, a sensor is selected that need not to have a high resolution (see the note below) but that can observe the whole range, which is to be observed around the vehicle, as quickly as possible. On the other hand, the fine observation sensor 16 is typically a sensor, such as a millimeter wave radar (phased array radar, etc.) or a lidar, that magnetically or optically scans and observes the situation around the vehicle but may be a camera that captures the situation around the vehicle. As will be understood from the description below, the purpose of the fine observation sensor 16 to more accurately observe and recognize an object that is included in the objects detected by the coarse observation sensor 14 during the extensive observation around the vehicle and that is to be observed more in detail for use in driving assistance control or autonomous driving control. Therefore, as the fine observation sensor 16, a sensor is selected that, when observing a particular area or object, can observe the particular area or object at a resolution high enough to meet the purpose of the observation and recognition. The coarse observation sensor 14 and the fine observation sensor 16 for practical use may be appropriately selected according to the design of the vehicle, the cost, etc. Note that the visual field or the observation range of the coarse observation sensor 14 and the fine observation sensor 16 may be appropriately set so that the area in front of, to the sides of, and behind the vehicle can be observed.

[0027] (Note) The resolution of the coarse observation sensor 14 and the fine observation sensor 16 may be spatial resolution or angular resolution. The spatial resolution represents the minimum of the distance between two points at which the points can be distinguished in the space observed by the sensor. The angular resolution represents the minimum of the angle between two points at which the points can be distinguished in the visual field observed by the sensor. A high resolution means that the distinguishable distance or angle between two points is small.

[0028] The observation control device 12, which may be implemented by a computer, may include a computer or a driving circuit that has a CPU, a ROM, a RAM, and an input/output port device that are interconnected by a standard, bidirectional common bus. The configuration and the operation of each of the components of the observation control device 12, which will be described later, may be implemented by the operation of the computer that works according to a program.

[0029] Referring to FIG. 1B, and more specifically, the observation control device 12 may receive not only the observation data from the coarse observation sensor 14 and the fine observation sensor 16 but also the information indicating the vehicle motion state such as the vehicle speed (calculated from wheel speed, etc.), steering angle, yaw rate, etc. Alternatively, though not shown, the observation control device 12 may acquire the moving distance and the turning angle (change in yaw rate) of the vehicle from the GPS information (The information indicating the vehicle motion state, such as the vehicle speed, steering angle or yaw rate, moving distance, turning angle, etc. of the vehicle, is generically called “vehicle motion state information”). When the observation data (for example, brightness signal) from the coarse observation sensor 14 is received by the observation control device 12, its data format is first converted to a format in which the object in the observation range is recognizable, for example, converted to the image format (image generation unit). The observation range of data obtained at this time may be the whole range around the vehicle to be observed. After that, in the observation range of the data having the format such as the above-described image format, the object recognition unit detects and recognizes objects, with the result that there positions, presence ranges, types, speeds, and moving directions, etc. (seen from the vehicle) are detected at the resolution of the coarse observation sensor 14. After that, the high-accuracy observation object future presence area prediction unit determines a high-accuracy observation object (object to be observed more accurately) from among the detected objects in the manner that will be described later. Then, using the detection position, type, and the vehicle motion state information, the high-accuracy observation object future presence area prediction unit predicts the object future presence area, which is the expected area to which the high-accuracy observation object is expected to be present in the future (at the time when the high-accuracy observation object is observed by the fine observation sensor 16). Note that there may be a plurality of high-accuracy observation objects and a plurality of object future presence areas.

[0030] When the high-accuracy observation object and the object future presence area are determined as described above, the information is given to the fine observation sensor 16 so that the fine observation sensor 16 can observe the object at a higher resolution in the object future presence area. The observation data obtained during this observation (such as reflected-wave signal intensity) is sent to the observation result processing unit of the observation control device 12 and, in this unit, its data format is converted to the data format in which the object can be recognized. Then, the object recognition unit detects and recognizes the object in the object future presence area using the data format in which the object can be recognized, with the result that its position, presence range, type, speed, moving direction, etc. (seen from the vehicle) are detected at the resolution of the fine observation sensor 16.

[0031] In this way, the extensive information on the situation around the vehicle, detected and recognized by the coarse observation sensor 14, and the more accurate recognition information on a high-accuracy observation object, obtained during the observation by the fine observation sensor 16, are sent to the observation result integration/output unit. From that unit, the information on the situation around the vehicle and the information on the object may be sent to the corresponding control device so that the information will be used for driving assistance control and autonomous driving control. System Operation

[0032] (1) Overview

[0033] As mentioned in “Summary of the Disclosure”, when using the observation information on the situation around a vehicle for driving assistance control or autonomous driving control, it is preferable that the information on an object in the observed range, such as the position or presence range, type, speed, and moving direction, be detected and recognized with a higher accuracy. However, the higher the accuracy of the observation, the longer the time required for the observation. Therefore, it is sometime impossible to secure sufficient time for accurately observing the whole range to be observed around the vehicle, for example, while the vehicle is traveling. On the other hand, an object such as that used for driving assistance control or autonomous driving control, for which high-accuracy information is required, is present usually in a part of the range to be observed around the vehicle. This means that, once the approximate position of an object for which high-accuracy information is desired can be recognized, it is, in some cases, sufficient for high-accuracy observation to be performed only for the object for which such high-precision information is desired. Considering this fact, the observation is performed in the system in this embodiment as described in Japanese Patent Application No. 2020-71587. That is, in consideration of the speed of the motion of the vehicle, the observation of the whole observation range around the vehicle is performed quickly at a resolution high enough to obtain the information such as the presence/absence, position or presence range, type, speed, and moving direction of the objects in the observation range, while the observation at a high resolution is performed only for an object that need be observed with high accuracy. This reduces the whole observation time and, at the same time, gives high-accuracy information suitable for driving assistance control or autonomous driving control.

[0034] However, in the observation described above, it takes a certain amount of time from the time the observation of the whole observation range around the vehicle is performed using the coarse observation sensor to recognize the objects in the observation range and to identify a high-accuracy observation object to the time the observation using the fine observation sensor is started. During this period of time, the high-accuracy observation object or the vehicle may move to another position or change the direction. For example, as schematically shown in FIG. 3A to FIG. 3B and in FIG. 4A to FIG. 4B, the high-accuracy observation object (ob) moves from the position, seen from the vehicle 10 and identified in the observation range of the coarse observation sensor, to another position. In such a case, even if the observation by the fine observation sensor is performed at the position in the observation range identified by the coarse observation sensor, the high-accuracy observation object may not be performed (FIG. 3C, FIG. 4C). To address this problem, the system in this embodiment is configured to observe a high-accuracy observation object more reliably. That is, after the high-accuracy observation object is identified in the observation range identified by the coarse observation sensor, the expected position (object future presence area) of the high-accuracy observation object at the time the observation of the fine observation sensor is performed is predicted or estimated. Then, the observation by the fine observation sensor is performed in the predicted or estimated object future presence area.

[0035] (2) Operation of Observation Processing

[0036] Referring to FIG. 2A and FIG. 2B, the general operation of the observation processing will be described. In the operation of the system in this embodiment, the following processing is performed sequentially:

(i) Observation of the whole observation range around the vehicle by the coarse observation sensor (step 1)
(ii) Recognition of objects in the observation range (step 2)
(iii) Determination of a high-accuracy observation object (step 3)
(iv) Prediction of the future presence area of the high-accuracy observation object (steps 4 to 6)
(v) Observation of the object future presence area by the fine observation sensor (step 7) (vi) Recognition of an object in the object future presence range (step 8)
(vii) Output of the observation result (step 9) The above processing will be described below sequentially.

[0037] (i) Observation of the Whole Observation Range Around the Vehicle by the Coarse Observation Sensor (Step 1)

[0038] As mentioned above, the observation by the coarse observation sensor may be typically performed by capturing an image by the camera in the usual manner as quickly as possible in the area to be observed around the vehicle (in front of, to the right and left of, and behind the vehicle, respectively). The resolution required in this case may be a resolution high enough to identify the presence or absence of objects in the observation range and to identify the positions or presence ranges of the objects at a certain degree of accuracy. The data obtained by the coarse observation sensor (usually brightness data or intensity data) may be generated as two-dimensional (or three-dimensional) image data by the image generation unit.

[0039] (ii) Recognition of Objects in the Observation Range (Step 2)

[0040] In the image data obtained by the image generation unit, the images of objects (such as other vehicles, roadside buildings, walls, fences, guardrails, poles, parked vehicles, pedestrians (pedestrians, bicycles), road ends, road markings (white lines, yellow lines), and traffic lights) are recognized, and the positions or presence ranges of those objects are detected (at the resolution of the coarse observation sensor). In addition, as will be described later, the type of an object described above or the moving speed and moving direction (seen from the vehicle) of an object may be detected in this step. A plurality of objects may be detected in the observation range. An object may be recognized and detected using any image recognition technique.

[0041] (iii) Determination of a High-Accuracy Observation Object (Step 3)

[0042] An object that is included in the objects recognized in the observation range of the coarse observation sensor in step 2 and is to be observed at a particularly high accuracy may be determined by any method according to the use purpose of the observation result. For example, when the observation result is used for driving assistance control for collision avoidance or is used for autonomous driving control, an object that will have a large impact on later driving may be selected as a high-accuracy observation object by referring to the distance from the vehicle to the object, the moving direction of the object, and the type of the object. In one mode, one possible method is that a threat level is given to each of the objects recognized in the observation range. In giving this threat level, it is assumed that the threat of an object to the traveling of the vehicle increases (the need for attention increases) as the distance to the vehicle is shorter, as the moving direction is more likely to intersect the traveling path of the vehicle, or as the moving speed is higher; alternatively, it is assumed that the threat increases in the order of a stationary object, another vehicle, and a person. After that, the threat levels of each object are totaled, the objects are ranked according to the threat level, and the high-accuracy observation objects to be observed preferentially are determined in the descending order of the rank. A plurality of objects may be selected as a high-accuracy observation object in a certain observation range.

[0043] (iv) Prediction of the Future Presence Area of the High-Accuracy Observation Object (Steps 4 to 6)

[0044] After a high-accuracy observation object is determined as described above, the expected position of the high-accuracy observation object in the future, more specifically, the expected position when observation is performed by the fine observation sensor (that is, the object future presence area) is predicted. The prediction of the object future presence area may be achieved in various ways, for example, as follows.

[0045] (a) Prediction of the Object Future Presence Area by Referring to the Type of the High-Accuracy Observation Object

[0046] In one mode, the object future presence area may be predicted according to the type of the high-accuracy observation object. In short, the movable range of the high-accuracy observation object from the observation position is calculated in this case based on the moving speed predicted according to the type of the high-accuracy observation object, and the calculated movable range is predicted as the object future presence area. In addition, the predicted position of the vehicle at the time when the observation is performed by the fine observation sensor may be calculated using the vehicle motion information, the object future presence area may be corrected to the position seen from the predicted position of the vehicle and, in addition, the angular range seen from the vehicle for observing the object future presence area may be determined.

[0047] More specifically, first, referring to FIG. 3A, when the object ob, recognized in the observation range cs of the coarse observation sensor 14, is determined in step 3 as a high-accuracy observation object, the center position X.sub.ob of the high-accuracy observation object ob is determined as follows based on the distance r.sub.o (in the X-Y coordinate space fixed to the vehicle) and the direction θ.sub.o (angle from the X axis) of the high-accuracy observation object ob seen from the vehicle 10:

X.sub.ob=(r.sub.o cos θ.sub.o,r.sub.o sin θ.sub.o)  (1)

[0048] Now, let Δt be the length of time from the coarse observation sensor observation time t1 to the fine observation sensor observation time t2 when the maximum moving speed v.sub.max assumed for the high-accuracy observation object ob is used. Then, the expected position of the high-accuracy observation object ob after an elapse of time Δt is on a circle with a radius of v.sub.max Δt and the center at the position X.sub.ob or is the inside of the circle as shown in FIG. 3A. In this case, it is thought that the maximum moving speed v.sub.max of the high-accuracy observation object ob is determined according to the type of the high-accuracy observation object ob, that is, according what is the high-accuracy observation object ob (a person, a bicycle, an automobile, a motorcycle, etc.) For example, the maximum moving speed v.sub.max may be assumed as follows:

[0049] Person: 0 km/h (It is thought that a person hardly moves)

[0050] Bicycle: 20 km/h

[0051] Automobile: 100 km/h

[0052] Motorcycle: 80 km/h

Thus, the range to which the high-accuracy observation object ob will move in the future is predicted to be on or inside the following circle (step 4):

{(Y−r.sub.o cos θ.sub.o).sup.2+(X−r.sub.o sin θ.sub.o).sup.2}.sup.1/2=v.sub.maxΔt  (2)

[0053] That is, the object future presence range that varies in size depending on the type of the high-accuracy observation object may be predicted. When the motion of the vehicle 10 is not taken into consideration, this range to which the high-accuracy observation object ob will move in the future may be used as the object future presence area.

[0054] In addition, when the motion information on the vehicle 10 is acquired (step 5) and, based on the acquired motion information on the vehicle 10, the range to which the high-accuracy observation object ob, which has been predicted as described above, will move in the future is corrected, the accuracy of the object future presence area is improved. More specifically, when the vehicle 10 is moving, for example, at the vehicle speed V.sub.c, initial yaw rate γ.sub.o, and yaw angle acceleration a.sub.c as shown in FIG. 3B, the yaw angle Ψ.sub.f and the position X.sub.vf (x.sub.vf, y.sub.vf) of the vehicle 10 at time t2 are expressed in the coordinates before the movement as follows:

Ψ.sub.f=γ.sub.oΔt+a.sub.cΔt.sup.2/2  (3a)

x.sub.vf=∫V.sub.c.Math.cos Ψf(t)dt  (3b)

y.sub.vf=∫V.sub.c.Math.sin Ψf(t)dt  (3c)

(The integration interval is [0, Δt].)

[0055] Here, the center position X.sub.obf (x.sub.obf, y.sub.obf) of the high-accuracy observation object ob seen from the vehicle 10 at time t2 is expressed as follows by converting the coordinates from the X-Y coordinates to the Xf-Yf coordinates using expressions (3a) to (3c).

[00001]$\begin{array}{cc}\left[\mathrm{Expression}1\right]& \\ \left(\begin{array}{c}{x}_{obf}\\ y_{obf}\end{array}\right)=\left(\begin{array}{cc}\cos {\Psi }_f& -sin{\Psi }_f\\ \sin {\Psi }_f& \cos {\Psi }_f\end{array}\right)\left(\left(\begin{array}{c}{r}_o\cos {\theta }_o\\ r_o\sin {\theta }_o\end{array}\right)-\left(\begin{array}{c}{x}_{vf}\\ y_{vf}\end{array}\right)\right)& \left(4\right)\end{array}$

[0056] Thus, the object future presence area of the high-accuracy observation object ob seen from the vehicle 10 after time t.sub.o is predicted to be on the following circle:

{(Y.sub.f−y.sub.obf).sup.2+(X.sub.f−x.sub.obf).sup.2}.sup.1/2=v.sub.maxΔt  (5)

[0057] That is, when the motion of the vehicle is taken into consideration, the object future presence area of the high-accuracy observation object ob seen from the vehicle 10 moves from the circle W, which is the presence area at the time of observation by the coarse observation sensor, to the circle W_f, as shown in FIG. 3C. This means that, when the motion of the vehicle is taken into consideration, the area on and inside the circle W_f is predicted as the object future presence area (step 6).

[0058] Thus, as shown in FIG. 3D, the range to be observed by the fine observation sensor 16 is the angular range ps in which the circle W_f can be viewed, with the angular range ps between angle φ.sub.max and angle φ.sub.min when the Xf-Yf coordinates of the circle W_f are converted to the polar coordinates. The angular coordinates φ on the circle in expression (5) and the range of X.sub.f are given by the following expression:

[00002]$\begin{array}{cc}\left[\mathrm{Expression}2\right]& \\ \varphi ={\tan }^{-1}\left(\frac{Y_f}{X_f}\right)={\tan }^{-1}\left(\frac{\pm \sqrt{v_{\max }^2\Delta t^2-{\left(X_f-x_{obf}\right)}^2}+y_{obf}}{X_f}\right)& \left(6\right)\\ x_{\mathrm{obf}}-v_{\max }\Delta t\leqq X_f\leqq x_{\mathrm{obf}}+v_{\max }\mathrm{\Delta t}& \left(6a\right)\end{array}$

Therefore, the angular range ps to be observed by the fine observation sensor 16 can be determined by calculating the maximum value φ.sub.max and the minimum value φ.sub.min of the angular coordinates φ in expression (6) in the range indicated by the range (6a).

[0059] If the moving distance x.sub.vf, y.sub.vf or the turning angle Ψ.sub.f of the vehicle during time Δt can be obtained in real time from the GPS information etc. when predicting the object future presence area as described above, those obtained values may be used in place of the calculations in expressions (3a) to (3c). In addition, if the turning angle Ψ.sub.f during time Δt is negligible, the coordinate rotation calculation in expression (4) need not be performed. If v.sub.max=0, the object future presence area may be predicted as a range of the object size d.

[0060] (b) Prediction of the Object Future Presence Area Using the Speed of a High-Accuracy Observation Object

[0061] In another mode, when the speed and the moving direction of an object seen from the vehicle can be detected during the observation by the coarse observation sensor (when the relative speed in the x direction and the relative speed in the y direction of the object can be detected separately (step 2 in FIG. 2B)), the object future presence area may be predicted in consideration of the motion of the high-accuracy observation object and the motion of the vehicle. To put it briefly, in this case, based on the speed of the high-accuracy observation object (relative speed in the x direction and the relative speed in the y direction) and the turning angle of the vehicle in the coordinate space fixed to the vehicle 10 (obtained in step 5 in FIG. 2B), the expected position from that observation position is predicted as the object future presence area (step 6).

[0062] More specifically, when the object ob recognized in the observation range cs of the coarse observation sensor 14 is determined as a high-accuracy observation object as shown in FIG. 4A (as in (a)), the position X.sub.ob of the high-accuracy observation object ob is determined from the distance r.sub.o and the direction θ.sub.o (angle from the X axis) of the high-accuracy observation object ob seen from the vehicle 10 (in the X-Y coordinate space fixed to the vehicle) in the same way as in expression (1). Now assume that the high-accuracy observation object ob is moving relative to the vehicle 10 at the speed of v(v.sub.x, v.sub.y). Then, as shown in FIG. 4B, during the period of time Δt from time t1 the observation by the coarse observation sensor is performed to time t2 the observation by the fine observation sensor is performed, the high-accuracy observation object ob moves by (v.sub.xΔt, v.sub.yΔt) from the position indicated in expression (1) (moves to the position of ob_f in the figure). During this period, if the vehicle 10 has turned at the initial yaw rate of γ.sub.o and the yaw angular acceleration a.sub.c, the yaw angle Ψf of the vehicle 10 at time t2 is given by expression (3a). Therefore, the position ob_f (x.sub.f, y.sub.f) of the high-accuracy observation object ob seen from the vehicle 10 at time t2 is expressed as follows by converting the position from the X-Y coordinates to the Xf-Yf coordinates using the yaw angle Ψ.sub.f.

[00003]$\begin{array}{cc}\left\{\mathrm{Expression}3\right]& \\ \left(\begin{array}{c}{x}_{obf}\\ y_{obf}\end{array}\right)=\left(\begin{array}{cc}\cos {\Psi }_f& -\sin {\Psi }_f\\ \sin {\Psi }_f& \cos {\Psi }_f\end{array}\right)\left(\begin{array}{c}{r}_o\cos {\theta }_o+v_x\mathrm{\Delta t}\\ r_o\sin {\theta }_o+y_y\mathrm{\Delta t}\end{array}\right)& \left(7\right)\end{array}$

[0063] Thus, at observation time t2 of the fine observation sensor, the high-accuracy observation object ob will move to the position ob_f shown in FIG. 4C when seen from the vehicle 10. Therefore, the object future presence area of the high-accuracy observation object ob can be predicted as the range of the high-accuracy observation object ob having the size of d and centered on the position ob_f (x.sub.obf, y.sub.obf). As shown in FIG. 4D, the range to be actually observed by the fine observation sensor 16 is the angular range ps in which the side d of the high-accuracy observation object ob_f can be viewed. The angular range ps is a range from the angle φ.sub.min to y.sub.max when the Xf-Yf coordinates are converted to the polar coordinates. Here, the polar coordinates (r.sub.f, φ.sub.f) of the center position of the high-accuracy observation object ob_f are as follows:

r.sub.f=(x.sub.obf.sup.2+Y.sub.obf.sup.2).sup.1/2  (8a)

φ.sub.f=tan.sup.−1(y.sub.obf/x.sub.obf)  (8b)

[0064] Therefore, the angular range ps to be observed by the fine observation sensor 16 is determined as a range between the following angles:

φ.sub.min=φ.sub.f−tan.sup.1(d/(2r.sub.f))  (9a)

φ.sub.max=φ.sub.f+tan.sup.−1(d/(2r.sup.f))  (9b)

[0065] If the turning angle Ψ.sub.f of the vehicle during time Δt can be obtained in real time from the GPS information etc. when predicting the object future presence area as described above, those obtained values may be used in place of the calculations in expression (3a). In addition, if the turning angle Ψ.sub.f during time Δt is negligible, the coordinate rotation calculation in expression (7) need not be performed.

[0066] (v) Observation of the Object Future Presence Area by the Fine Observation Sensor (Step 7)

[0067] When the object future presence area is predicted and the angular range ps in which the predicted object future presence area can be viewed is determined as described above, the observation is performed by the fine observation sensor at an angle in the angular range ps. The resolution required in this step may be high enough for use as the information acceptable or satisfactory for driving assistance control or autonomous driving control.

[0068] (vi) Recognition of an Object in the Object Future Presence Range (Step 8)

[0069] The data obtained by the fine observation sensor (usually, intensity data or brightness data) may be sent to the observation result processing unit for converting it into a data format that allows the object to be recognized. After that, the object recognition unit recognizes the high-accuracy observation object based on the data obtained by the observation result processing unit. More specifically, the position or presence range and the type are identified, and the moving speed and the moving direction are detected, at an accuracy higher than that of the object obtained by the coarse observation sensor.

[0070] (vii) Output of the Observation Result (Step 9)

[0071] The information on the object recognized/detected through the observation by the coarse observation sensor and the fine observation sensor as described above may be integrated, as appropriate, and output to the corresponding control devices for use in driving assistance control and autonomous driving control.

[0072] Thus, as described in the above example, the system in this embodiment is an in-vehicle sensor system for observing the area around the vehicle using the coarse observation sensor and the fine observation sensor. This in-vehicle sensor system predicts the position, or the presence area, of an object to be observed by the fine observation sensor in consideration of the motion of the object to be detected by the sensor or the motion of the vehicle itself and performs observation by the fine observation sensor at the predicted position or in the predicted presence area. Therefore, it is expected that the observation of a high-accuracy observation object will be performed more reliably. The information on the area around the vehicle, acquired by the system in this embodiment, may be advantageously used in driving assistance control and autonomous driving control of the vehicle.

[0073] Although the above description has been made in connection with the embodiments of the present disclosure, many changes and modifications can be easily made by those skilled in the art. It is apparent that the present disclosure is not limited to the embodiments exemplified above but may be applied to various devices without departing from the concept of the present disclosure.