IMAGE PROCESS BASED, DYNAMICALLY ADJUSTING VEHICLE SURVEILLANCE SYSTEM FOR INTERSECTION TRAFFIC

Abstract

A dynamically adjusting surveillance system provides a monitor, a sequence of views that include a key view. The sequence of views is captured by a camera. The monitor can display a portion of the camera view. The system is configured such that the displayed portion of the camera view includes the key view to a driver of the vehicle. The key view can be selected to be a rearward view when the vehicle is not turning, and a view of cross traffic from a side opposite which a turn is made at an intersection.

Claims

1. A dynamically adjusting surveillance system comprising: at least one camera configured to capture a sequence of views containing a key view; and a controller and memory unit configured to receive the sequence of views, the controller and memory unit further configured, based on the sequence of views, to measure changes in angular position of a vehicle using an image processing technique; wherein the controller and memory unit selects portions of the sequence of views based on a measured change in angular position of the vehicle, wherein the selected portions contain the key view.

2. The dynamically adjusting surveillance system of claim 1, wherein the view of cross traffic is in a constant direction during the turn.

3. The dynamically adjusting surveillance system of claim 1, further comprising a monitor, wherein the controller and memory unit is further designed to display the selected portions of the sequence of views on the monitor.

4. The dynamically adjusting surveillance system of claim 1, wherein the key view comprises a view of a road section toward behind the vehicle on the left side before a turn is initiated.

5. The dynamically adjusting surveillance system of claim 1, wherein the key view comprises a view of a road section toward behind the vehicle on the right side before a turn is initiated.

6. The dynamically adjusting surveillance system of claim 1, wherein: two views in the sequence of views are indexed by integers designated view(1) and view(2); and a change in angular position of the vehicle at index 2 is calculated by taking a portion, Q1, of pixels in a camera view, view(1), at index 1, then finding a best representative, Q2, in the camera view, view(2) at index, 2, and measuring a displacement from Q1 portion to Q2 portion and driving the angular position change based on the displacement.

7. The dynamically adjusting surveillance system of claim 6, wherein the portion, P1, is at close proximity of a vanishing point in the view(1).

8. The dynamically adjusting surveillance system of claim 1, further comprising a Global Positioning System (GPS) providing GPS signals to the controller and memory unit, wherein, for a selection of the portion of the sequence of views, the controller and memory unit is further configured to incorporate the GPS signals.

9. The dynamically adjusting surveillance system of claim 1, further comprising a Global Positioning System (GPS) providing GPS signals to the controller and memory unit, wherein for signaling end of a turn, the controller and memory unit is further configured to incorporate the GPS signals.

10. The dynamically adjusting surveillance system of claim 1, further comprising a Global Positioning System (GPS) providing GPS signals to the controller and memory unit, wherein for signaling the start of a turn, the controller and memory unit is further configured to incorporate the GPS signals.

11. The dynamically adjusting surveillance system of claim 1, further comprising a turn signal switch, wherein the controller and the memory unit is further designed to start a turn when the turn signal switch is on.

12. The dynamically adjusting surveillance system of claim 1, wherein the controller and the memory unit is further designed to start a turn after total changes in angular position exceeds a predetermined angle

13. The dynamically adjusting surveillance system of claim 1, wherein the controller and the memory unit is further designed to start a turn after both following conditions are satisfied: 1) total changes in angular position exceeds a predetermined angle; and 2) the vehicle is at close proximity of an intersection.

14. The dynamically adjusting surveillance system of claim 13, wherein the condition 2) is accomplished by configuring the controller and memory unit to detect the presence and absence of a street traffic line.

15. A dynamically adjusting surveillance system comprising: at least one camera configured to capture a sequence of views containing a key view; a controller and memory unit configured to receive the sequence of views, the controller and memory unit further configured, based on the sequence of views, to measure changes in angular position of a vehicle using an image processing technique; and a monitor, wherein the controller and memory unit is further designed to display the selected portions of the sequence of views on the monitor; wherein the controller and memory unit selects portions of the sequence of views based on a measured change in angular position of the vehicle, wherein the selected portions contain the key view; wherein the key view is a rearward view when the vehicle is not turning and is a view of cross traffic from a side opposite which a turn is made at an intersection; and wherein the view of cross traffic is in a constant direction during the turn.

16. The dynamically adjusting surveillance system of claim 15, wherein: two views in the sequence of views are indexed by integers designated view(1) and view(2); and a change in angular position of the vehicle at index 2 is calculated by taking a portion, Q1, of pixels in a camera view, view(1), at index 1, then finding a best representative, Q2, in the camera view, view(2) at index, 2, and measuring a displacement from Q1 portion to Q2 portion and driving the angular position change based on the displacement.

17. The dynamically adjusting surveillance system of claim 15, further comprising: a Global Positioning System (GPS) providing GPS signals to the controller and memory unit, wherein, for a selection of the portion of the sequence of views, the controller and memory unit is further configured to incorporate the GPS signals; wherein for signaling end of a turn, the controller and memory unit is further configured to incorporate the GPS signals; and wherein for signaling the start of a turn, the controller and memory unit is further configured to incorporate the GPS signals.

18. The dynamically adjusting surveillance system of claim 15, further comprising a turn signal switch, wherein the controller and the memory unit is further designed to start a turn when to turn signal switch is on.

19. The dynamically adjusting surveillance system of claim 15, wherein the controller and the memory unit is further designed to start a turn after a first one or a first and second ones of following conditions are satisfied: 1) total changes in angular position exceeds a predetermined angle; and 2) the vehicle is at close proximity of an intersection.

20. The dynamically adjusting surveillance system of claim 19, wherein the condition 2) is accomplished by configuring the controller and memory unit to detect the presence and absence of a street traffic line.

21. A dynamically adjusting surveillance system comprising: at least one camera configured to capture a sequence of views containing a key view; and a controller and memory unit configured to receive the sequence of views, the controller and memory unit further configured, based on the sequence of views, to measure changes in angular position of the at least one camera using an image processing technique.

22. The dynamically adjusting surveillance system of claim 21, wherein: two views in the sequence of views are indexed by integers designated view(1) and view(2); and a change in angular position of the vehicle at index 2 is calculated by taking a portion, Q1, of pixels in a camera view, view(1), at index 1, then finding a best representative, Q2, in the camera view, view(2) at index, 2, and measuring a displacement from Q1 portion to Q2 portion and driving the angular position change based on the displacement.

23. The dynamically adjusting surveillance system of claim 22, wherein the portion, P1, is at close proximity of a vanishing point in the view(1).

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Some embodiments of the present invention are illustrated as an example and are not limited by the figures of the accompanying drawings, in which like references may indicate similar elements.

[0050] FIG. 1 illustrates conventional left and right side mirror views of a vehicle making a left turn;

[0051] FIG. 2 illustrates conventional left and right side mirror views of a vehicle making a right turn;

[0052] FIG. 3 illustrates a block diagram of an image processing based dynamically adjusting surveillance system in accordance with an exemplary embodiment of the present invention;

[0053] FIG. 4A illustrates a left-side mirror view before a vehicle reaches an intersection;

[0054] FIG. 4B illustrates a left-side mirror view when a vehicle is just crossing an intersection;

[0055] FIG. 5 illustrates an application of the image process based, dynamically adjusting vehicle surveillance system for intersection traffic in accordance with an exemplary embodiment of the present invention;

[0056] FIG. 6 illustrates a summary of the overall operation of the image process based, dynamically adjusting vehicle surveillance system for intersection traffic, in accordance with an exemplary embodiment of the present invention;

[0057] FIG. 7 illustrates a block diagram of an image process based, dynamically adjusting vehicle surveillance system for intersection traffic, in accordance with an exemplary embodiment of the present invention;

[0058] FIG. 8 illustrates a summary of the overall operation of the image process based, dynamically adjusting vehicle surveillance system for intersection traffic, including a side information detector, in accordance with an exemplary embodiment of the present invention;

[0059] FIG. 9 illustrates an application of the image process based, dynamically adjusting vehicle surveillance system for intersection traffic, according to another exemplary embodiment of the present invention;

[0060] FIG. 10 illustrates an application of the image process based, dynamically adjusting vehicle surveillance system for intersection traffic, according to another exemplary embodiment of the present invention; and

[0061] FIG. 11 illustrates an application of the image process based, dynamically adjusting vehicle surveillance system for intersection traffic, according to another exemplary embodiment of the present invention.

[0062] Unless otherwise indicated illustrations in the figures are not necessarily drawn to scale.

[0063] The invention and its various embodiments can now be better understood by turning to the following detailed description wherein illustrated embodiments are described. It is to be expressly understood that the illustrated embodiments are set forth as examples and not by way of limitations on the invention as ultimately defined in the claims.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS AND BEST MODE OF INVENTION

[0064] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well as the singular forms, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and/or groups thereof.

[0065] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one having ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present disclosure and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0066] In describing the invention, it will be understood that a number of techniques and steps are disclosed. Each of these has individual benefit and each can also be used in conjunction with one or more, or in some cases all, of the other disclosed techniques. Accordingly, for the sake of clarity, this description will refrain from repeating every possible combination of the individual steps in an unnecessary fashion. Nevertheless, the specification and claims should be read with the understanding that such combinations are entirely within the scope of the invention and the claims.

[0067] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be evident, however, to one skilled in the art that the present invention may be practiced without these specific details.

[0068] The present disclosure is to be considered as an exemplification of the invention, and is not intended to limit the invention to the specific embodiments illustrated by the figures or description below.

[0069] Devices or system modules that are in at least general communication with each other need not be in continuous communication with each other, unless expressly specified otherwise. In addition, devices or system modules that are in at least general communication with each other may communicate directly or indirectly through one or more intermediaries.

[0070] A description of an embodiment with several components in communication with each other does not imply that all such components are required. On the contrary a variety of optional components are described to illustrate the wide variety of possible embodiments of the present invention.

[0071] The present invention is described using various examples. Each example describes various situations and embodiments where the system of the present invention may be applied. The examples are used to describe specific incidents in which the present invention may be useful but is not meant to limit the present invention to such examples.

EXAMPLE 1

[0072] Example 1 is explained referring to FIGS. 3 through 6.

[0073] This example relates to scenario A4—referring to a view in the left-side mirror during a right turn. More specifically, an image process based, dynamically adjusting vehicle surveillance system for intersection traffic of Example 1 alleviates the shortcomings of the left-side mirror during right turns as explained earlier.

[0074] Here, the key desired view is a view of a left-side mirror when the vehicle is not turning right at an intersection, and the desired view is a view of a road section opposite to that of the road section into which the vehicle is turning; that is, a view of the cross traffic from the left side when the vehicle is turning right at an intersection.

[0075] FIG. 3 gives a block diagram of the image process based, dynamically adjusting vehicle surveillance system for intersection traffic. The image process based, dynamically adjusting vehicle surveillance system for intersection traffic includes a camera 10, a controller and memory unit 20, and a monitor 30.

[0076] The camera 10 may be placed on a side mirror housing, the controller and memory unit 20 may be affixed behind the monitor 30, and the monitor 30 may be placed in one or more of the following locations: 1) it may replace a side mirror, 2) it may be placed anywhere a GPS system is placed inside a vehicle, and 3) if a monitor already exists in the vehicle, it may be configured to act as the monitor 30.

[0077] Referring to FIG. 3, the camera 10 is configured to capture a sequence of views that contain the desired key view. The sequence of views of the camera 10 can be indexed in a natural way: view(1), view(2), and so on. Each of the views of the camera 10 contains a view of a left-side mirror, and because the camera 10 has a relatively wide angle the view extends to other areas.

[0078] The controller and memory unit 20 is configured to receive the sequence of the views: view(1), view(2) and so on.

[0079] When the vehicle is not making a right turn at an intersection, the controller and memory unit 20 selects portions P1, P2, and so on, of the views, view(1), view(2), so on, respectively, such that Pi's are views of a left-side mirror. Then the controller and memory unit 20 displays portions P1, P2, and so on, on the monitor 30. The monitor 30 provides these views to the driver of the vehicle. These views are sent as video to the monitor.

[0080] When the vehicle is making a right turn at an intersection, the controller and memory unit 20 selects portions P1, P2, and so on, of the views, view(1), view(2), so on, respectively, such that Pi's are views of cross traffic from the left side. Since the camera 10 has a lens with relatively wide angle, when the vehicle is making a right turn, the views view(1), view(2), and so on, include a view of the cross traffic from the left side. Therefore, selection of portions P1, P2, and so on, to include views of cross traffic from the left side is feasible.

[0081] Next, the controller and memory unit 20 displays portions P1, P2, and so on, on the monitor 30. The monitor 30 provides these views for the driver of the vehicle.

[0082] How the Pi potions are selected will be explained below.

Angular Position Changes

[0083] The controller and memory unit 20 is configured to measure changes in angular position of the vehicle.

[0084] Consider the vehicle in two positions, first, the position of the vehicle when view(1) was captured by the camera 10, and second, the position of the vehicle when view(2) was captured.

[0085] In order to measure the angle between the first position and the second position of the vehicle the controller and memory unit 20 may perform the following.

[0086] The controller and memory unit 20 can take a small rectangular area, Q1, of size W×H pixels, where W, in pixels, is the width and H, in pixels, is the height of the area Q1 in the view(1) of the camera 10. Then, the controller and memory unit 20 finds a rectangular area, Q2, of the same size, in the view(2) of the camera 10 such that view(2), restricted to area Q2, is a best representation of view(1) restricted to area Q1.

[0087] Without loss of generality, one can assume that view(1) and view(2) are given in their RGB color format.

[0088] Then, view(1), restricted to area Q1, is given by a W×H×3 array, image1, where W is the width of the image, H is the height of the image; there are W×H pixels, and each pixel, (w,h), 0≦w≦W, 0≦h≦H, has 3 integers: image1(w,h,1)=r ∈ {0,1, . . . , 255}, image1(w,h,2)=g ∈ {0,1, . . . , 255}, and image1(w,h,3)=b ∈ {0,1, . . . , 255}, where r, g, and b are the levels of red, green, and blue of the pixel, respectively.

[0089] Similarly, view(2), restricted to area Q2, is given by a W×H×3 array, image2.

[0090] Now, the distance, D, between image1 and image2 can be defined as:

[00001]$D=\underset{w=0}{\overset{W}{.Math.}}.Math.\underset{h=1}{\overset{H}{.Math.}}.Math.\underset{c=1}{\overset{3}{.Math.}}.Math.\mathrm{abs}\left(\mathrm{image}.Math..Math.1.Math.\left(w,h,c\right)-\mathrm{image}.Math..Math.2.Math.\left(w,h,c\right)\right),$

[0091] where abs denotes absolute value. View(2), restricted to area Q2, is a best representation of view(1), restricted to Q1, if for any other area, Q3, Q3≠Q2:

[00002]$D\le \underset{w=0}{\overset{W}{.Math.}}.Math.\underset{h=1}{\overset{H}{.Math.}}.Math.\underset{c=1}{\overset{3}{.Math.}}.Math.\mathrm{abs}\left(\mathrm{image}.Math..Math.1.Math.\left(w,h,c\right)-\mathrm{image}.Math..Math.3.Math.\left(w,h,c\right)\right),$

[0092] where view(2), restricted to area Q3, is given by a W×H×3 array, image3.

[0093] Next, the controller and memory unit 20 finds a pair of integers (ph,pv) such that if Q1 is moved horizontally by ph and then vertically by pv, then Q2 is obtained.

[0094] Now the difference in the angular position of the vehicle from its position when view(1) was taken to its position when view(2) was taken may be defined as ph. The unit here is number of pixels. An angular position difference of ph pixels may be stated in degrees (or radians) as follows: (angle of view horizontally of the camera 10, in degrees)*ph/(width of a view of the camera 10 in pixels).

[0095] Unless the vehicle is moving very fast, the difference in the angular position, ph, is small. Therefore the search for Q2 in view(2) can be reduced to the search for Q2 close to Q1 in view(2).

[0096] The difference in the angular position of the vehicle can be defined using other shaped areas instead of rectangle. The rectangle area was chosen as an example.

[0097] It should be noted that the small area Q1 in the view(1) of the camera 10 may be selected at close proximity of a vanishing point of view(1).

[0098] For all integers, n, n>1, the difference in the angular position of the vehicle from its position when view(n) was taken to its position when view(n+1) was taken is defined similar to case n=1 above.

[0099] How a right turn is detected will be explained below.

[0100] The controller and memory unit 20 is configured to detect the following two conditions.

[0101] 1) Total changes in the angular position of the vehicle exceeds in magnitude a predetermined angle, alpha1, for example alpha1=45 degrees, and

[0102] 2) A change in the street lines indication entering an intersection. For example, FIG. 4A shows the left-side mirror view before the vehicle 4 reaches an intersection and FIG. 4B shows the left-side mirror view when the vehicle 4 is just crossing an intersection. A street line 40 is shown in FIG. 4B. The controller and memory unit 20 may, by noticing the line 40, detect the event that the vehicle is entering an intersection.

[0103] A right turn by the vehicle may be detected by the controller and memory unit 20 when one or both conditions above are detected.

[0104] The magnitude of the total changes in the angular position of the vehicle may grow without the vehicle making a right turn. For example, it may grow as the vehicle moves along a bendy road that gradually changes its direction, for example from toward north to toward east. This type of growth of the total changes in the angular position of the vehicle is not desirable since it may confuse the controller and memory unit 20 thinking the vehicle is in the process of a right turn. A good method that counters undesirable growth is to, for the purpose of detecting right turn, keep only the total changes over the last M views (or the last T seconds), where M (or T) is predetermined.

[0105] The condition 2) above prevents a sharp turn that is not part of a right turn to be marked as a right turn by the controller and memory unit 20. Nevertheless, since the vehicle does not encounter many sharp turns during normal driving conditions, the second condition may be eliminated. However, it has been kept in this example for extra accuracy.

How to Select Portions (P1, P2, and so on)

[0106] When a right turn is not detected, more specifically, the controller and memory unit 20 selects P1=PA, P2=PA, so on, where PA is a predetermined area in the view of the camera 10 that corresponds to a view of a typical left-side mirror.

[0107] When a right turn is detected, more specifically, the controller and memory unit 20 first selects P1=PB, where PB is a predetermined area in the view(1) of the camera 10 that corresponds to a view of a typical cross traffic from left side from a vehicle that has completed alpha1 degrees rotation into its right turn.

[0108] After receiving view(2), the controller and memory unit 20 measures the change in angular position of the vehicle from position in view(1) to position in view(2). This change can be denoted by ph2. Next, the controller and memory unit 20 defines

[0109] P2=SHIFT(P1,ph2), that is the portion P2 is P1 shifted by ph2 pixels. If ph2<0, then the shift is to the left, but if ph2>0, then the shift is to the right.

[0110] Then, the controller and memory unit 20 repeats the procedure, for the next view.

Boundaries

[0111] The image process based, dynamically adjusting vehicle surveillance system for intersection traffic of Example 1 may set a right limit portion, PR, and not allow portions pass to the right of PR by ignoring ph contributions that would push the portion pass to the right of PR. Similarly, the image process based, dynamically adjusting vehicle surveillance system for intersection traffic of Example 1 may set a left limit portion, PL, and not allow portions pass to the left of PL by ignoring ph contributions that would push the portion pass to the left of PL.

[0112] Therefore, the monitor 30 provides a driver of the vehicle with the desired key views, both while making right turns and while not making right turns. In addition, while making a right turn, the monitor 30 provides a view that is generally in a consistent direction throughout the turn.

[0113] FIG. 5 depicts an application of the image process based, dynamically adjusting vehicle surveillance system for intersection traffic of Example 1. The automobile 4 is depicted in three positions: facing north at the intersection 5 (position 22), facing north-east in the intersection 5 (position 25), and facing east past the intersection 5 (position 26). During most of the right turn, the left-side mirror 2 shows the bottom left corner of the intersection. Instead, the monitor 30 shows a view of the cross traffic from west in a generally consistent manner. The view of the monitor 30 does not rotate with the automobile 4, hence providing useful surveillance. In FIG. 5, the view of the cross traffic from west includes a view of an automobile 6.

How to Terminate a Right Turn

[0114] The controller and memory unit 20 ends a right turn and starts displaying a left-side mirror view on the monitor 30 when at least one of the following conditions is detected: 1) if street lines indicate end of a right turn, for example if a dashed white line is detected lining up with the vehicle; 2) if a selected portion at time n, during a right turn equals PA; 3) if a right turn takes longer than a predetermined time, N1; and 4) if the total changes in the angular position of the vehicle in the last N2 indices is less than alpha2, where N2 and alpha2 are predetermined values.

[0115] After detecting one of the above conditions, if the last portion, P, is not already the same as PA, then the controller and memory unit 20 gradually moves the portions to position PA.

[0116] Anyone of the above conditions may be used by itself without including the other conditions. To increase accuracy one may combine the above conditions or add more conditions.

Flow Chart

[0117] Overall operation of the image process based, dynamically adjusting vehicle surveillance system for intersection traffic of Example 1 may be summarized using FIG. 6 and Table 1.

[0118] The block diagram of FIG. 6 shows, in more detail, how changes in the angular position, ph, the total changes in the angular position, tph, and the partial changes in the angular position, tNph and tMph, may be computed, and how a “cross traffic line detector” and a “street dashed/solid line detector” may fit in a hardware or software implementation of the image process based, dynamically adjusting vehicle surveillance system for intersection traffic of Example 1.

[0119] Boxes labeled 50, 51, 52, 53, 54, and 55 are delay operators. The input to the box 50 is view(n), at time index n. The output of the box 50 is view(n−1), at time index n.

[0120] The views, view(n) and view(n−1), are input to a box 60 named “change in angular position”. The box 60 performs the procedure given above and computes the change in angular position, and it outputs, ph.

[0121] The box 51 and an adder 63 generate a running sum of the changes in the angular position of the vehicle. The input of the box 51 is the total changes in the angular position, tph.

[0122] On the other hand, a subtraction operator 64 together with N delay boxes, of which three are shown, generate the sum of the changes in the angular position over the last N time indices, tNph. The delay boxes shown are 52, 53, and 54.

[0123] Also a subtraction operator 67 together with M delay boxes, of which four are shown, generate the sum of the changes in the angular position over the last M time indices, tMph. The delay boxes shown are 52, 53, 54, and 55.

[0124] The view, view(n), is input to a box 61 named “cross traffic line detector”. The box 61 generates a binary output indicating whether a cross traffic line is detected or not. The box 61 performs the task given earlier.

[0125] The view, view(n), is input to another box 62 named “street dashed/solid line detector”. The box 62 generates a binary output indicating whether a street dashed or solid line is detected lining up with the vehicle or not. The box 62 performs the task given earlier.

[0126] The rest of the operation of the image process based, dynamically adjusting vehicle surveillance system for intersection traffic of Example 1 may be described using Table 1.

[0127] Table 1 gives a finite state transition machine, FSTM, with three states: 1, 2, and 3.

[0128] If the state=1, it means the FSTM is processing a “not making a right turn” by the vehicle.

[0129] If the state=2, it means the FSTM, is processing a “making a right turn” by the vehicle.

[0130] If the state=3, it means the FSTM, is processing a termination to a “making a right turn” by the vehicle. In this state, the FSTM returns the selected portions, P's, to PA.

[0131] For a given row, the column 2 gives the current state of the FSTM.

[0132] For a given row, the column 3 is yes if one of the following 4 conditions is satisfied, otherwise it is a no.

[0133] Condition 1: A counter “con1” has reached N1.

[0134] Condition 2: The previous selected portion, PX, equals PA.

[0135] Condition 3: Referring to FIG. 6, the box 62 output is yes.

[0136] Condition 4: A counter “con2” has reached N2 AND abs(tNph) of FIG. 6 is less than alpha2.

[0137] For a given row, the column 4 is yes if the following condition is satisfied, otherwise it is a no.

[0138] Condition 5: Referring to FIG. 6, the box 61 output is yes AND tMph is greater than alpha1.

[0139] For a given row, the column 5 is yes if the following condition is satisfied, otherwise it is a no.

[0140] Condition 2: The previous selected portion, PX, equals PA.

[0141] For a given row, the column 6 gives the next state of the FSTM.

[0142] For a given row, the column 7 gives the next selected portion.

[0143] For a given row, the column 8 states whether the counter con1 should be incremented or reset to zero or put to sleep.

[0144] For a given row, the column 9 states whether the counter con2 should be incremented or reset to zero or put to sleep.

TABLE-US-00001 TABLE 1 1 2 3 4 5 6 7 8 9 1 1 N 1 PA NA NA 2 1 Y 2 PB 0 0 3 2 N 2 SHIFT(PX, ph; PA) add 1 add 1 4 2 Y Y 1 PA NA NA 5 2 Y N 3 SHIFT(PX, PH; PA) NA NA 6 3 N 3 SHIFT(PX, PH; PA) NA NA 7 3 Y 1 PA NA NA

[0145] The rows of Table 1 are defined as follows:

[0146] Row 1: If the current state is 1, then columns 3 and 5 are not relevant. If the column 4 is no (no right turn is detected), then the next state is 1 (column 6) and the next selected region is PA (corresponds to left side mirror view, column 7). The counter con1 is sleep (column 8) and the counter con2 is sleep (column 9).

[0147] Row 2: If the current state is 1, then columns 3 and 5 are not relevant. If the column 4 is yes (right turn is detected), then the next state is 2 (column 6) and the next selected region is PB (corresponds to cross traffic from the left side, column 7). The counter con1 is reset to zero (column 8) and the counter con2 is reset to zero (column 9).

[0148] Row 3: If the current state is 2, then column 4 is not relevant. If the column 3 is no (tail of right turn is detected), then column 5 is not relevant. Then the next state is 2 (column 6) and the next selected region is PX=SHIFT(PX,ph; PA) (column 7), where SHIFT(PX,ph; PA) is a function that takes the region PX and moves it horizontally by ph pixels; if at any time during the move the region coincides with PA, then SHIFT(PX,ph; PA)=PA. The counter con1 is incremented (column 8) and the counter con2 is incremented (column 9).

[0149] Row 4: If the current state is 2, then column 4 is not relevant. If the column 3 is yes (tail of right turn is detected), and the previous selected portion, PX, was equal to PA (column 5), then the next state is 1 (column 6) and the next selected region is PX=PA (column 7). The counter con1 is put to sleep (column 8) and the counter con2 is put to sleep (column 9).

[0150] Row 5: If the current state is 2, then column 4 is not relevant. If the column 3 is yes (tail of right turn is detected), and the previous selected portion, PX, was not equal to PA (column 5), then the next state is 3 (column 6) and the next selected region is PX=SHIFT(PX,PH; PA) (column 7), where PH is a predetermined change in the angular position. The counter con1 is put to sleep (column 8) and the counter con2 is put to sleep (column 9).

[0151] Row 6: If the current state is 3, then columns 3 and 4 are not relevant. If the previous selected portion, PX, was not equal to PA (column 5), then the next state is 3 (column 6) and the next selected region is SHIFT(PX,PH; PA) (column 7). The counter con1 remains sleep (column 8) and the counter con2 remains sleep (column 9).

[0152] Row 7: If the current state is 3, then columns 3 and 4 are not relevant. If the previous selected portion, PX, was equal to PA (column 5), then the next state is 1 (column 6) and the next selected region is PX=PA (column 7). The counter con1 remains sleep (column 8) and the counter con2 remains sleep (column 9).

EXAMPLE 2

[0153] Example 2 is explained using FIGS. 3, 6 and 9.

[0154] This example relates to scenario A1—referring to a view in the right-side mirror during left turn. More specifically, an image process based, dynamically adjusting vehicle surveillance system for intersection traffic of Example 2 alleviates the shortcomings of the right-side mirror during left turns as explained earlier.

[0155] Here the key desired view is a view of a right-side mirror when the vehicle is not turning left at an intersection, and the desired view is a view of a road section opposite to that of the road section into which the vehicle is turning; that is, a view of the cross traffic from the right side when the vehicle is turning left at an intersection.

[0156] FIG. 3 gives a block diagram of the image process based, dynamically adjusting vehicle surveillance system for intersection traffic of Example 2. The image process based, dynamically adjusting vehicle surveillance system for intersection traffic of Example 2 includes the camera 10, the controller and memory unit 20, and the monitor 30.

[0157] Again, the controller and memory unit 20 is configured to receive the sequence of the views: view(1), view(2) and so on, that are captured by the camera 10.

[0158] When the vehicle is not making a left turn at an intersection, the controller and memory unit 20 selects portions P1, P2, and so on, of the views, view(1), view(2), and so on, respectively, such that Pi's are views of a right-side mirror. Then, the controller and memory unit 20 displays portions P1, P2, and so on, on the monitor 30. The monitor 30 provides these views to the driver of the vehicle.

[0159] When the vehicle is making a left turn at an intersection, the controller and memory unit 20 selects portions P1, P2, and so on, of the views, view(1), view(2), and so on, respectively, such that Pi's are views of a cross traffic from the right side.

[0160] Next the controller and memory unit 20 displays portions P1, P2, and so on, on the monitor 30. The monitor 30 provides these views for the driver of the vehicle.

[0161] How Pi potions are selected will be explained below.

Angular Position Changes

[0162] The controller and memory unit 20 is configured to measure the changes in angular position of the vehicle as in Example 1.

[0163] How a left turn is detected will be explained below.

[0164] The controller and memory unit 20 is configured to detect the following two conditions.

[0165] 1) Total changes in the angular position of the vehicle exceeds in magnitude a predetermined angle, alpha1, for example alpha1=45 degrees, and

[0166] 2) A change in the street lines indication entering an intersection.

[0167] A left turn by the vehicle is detected by the controller and memory unit 20 when one or both conditions above are detected.

[0168] The condition 2) above prevents a sharp turn that is not part of a left turn to be marked as a left turn by the controller and memory unit 20. Nevertheless, since the vehicle does not encounter many sharp turns during normal driving conditions, the second condition may be eliminated. However, it has been kept for extra accuracy.

How to Select Portions (P1, P2, and so on)

[0169] When a left turn is not detected, more specifically, the controller and memory unit 20 selects P1=PC, P2=PC, and so on, where PC is a predetermined area in the view of the camera 10 that corresponds to a view of a typical right-side mirror

[0170] When a left turn is detected, more specifically, the controller and memory unit 20 first selects P1=PD, where PD is a predetermined area in the view(1) of the camera 10 that corresponds to a view of a typical cross traffic from right side from a vehicle that has completed alpha1 degrees rotation into its left turn.

[0171] After receiving, view(2), the controller and memory unit 20 measures the change in angular position of the vehicle from position in view(1) to position in view(2). The change in angular position can be denoted by ph2. Next, the controller and memory unit 20 defines

[0172] P2=SHIFT(P1,ph2), that is the portion P2 is P1 shifted by ph2 pixels. If ph2<0, then the shift is to the left, but if ph2>0, then the shift is to the right.

[0173] Then, the controller and memory unit 20 repeats the procedure, for the next view.

[0174] Therefore, the monitor 30 provides a driver of the vehicle with the desired key views, both while making left turns and while not making left turns. In addition, while making a left turn, the monitor 30 provides a view that is generally in a consistent direction throughout the turn.

[0175] FIG. 9 depicts an application of the image process based, dynamically adjusting vehicle surveillance system for intersection traffic of Example 2. The automobile 4 is depicted in three positions: facing north at the intersection 5 (position 22), facing north-west in the intersection 5 (position 23), and facing west past the intersection 5 (position 24). During most of the left turn, the right-side mirror 1 shows the bottom right corner of the intersection. Instead the monitor 30 shows a view of the cross traffic from east in a generally consistent manner. The view of the monitor 30 does not rotate with the automobile 4, hence providing useful surveillance. In FIG. 9, the view of the cross traffic from east includes a view of an automobile 7.

How to Terminate a Left Turn

[0176] The controller and memory unit 20 ends a left turn and starts displaying a right-side mirror view on the monitor 30 when one of the following conditions is detected: 1) if street lines indicate end of a left turn, for example if a dashed/sold white line is detected lining up with the vehicle; 2) if a selected portion at time n, during a left turn equals PC; 3) if a left turn takes longer than a predetermined time, N1; and 4) if the total changes in the angular position of the vehicle in the last N2 indices is less than alpha2, where N2 and alpha2 are predetermined values.

[0177] After detecting one of the above conditions, if the last portion, P, is not already the same as PC, then the controller and memory unit 20 gradually moves the portions to position PC.

[0178] Anyone of the above conditions may be used by itself without including the other conditions. To increase accuracy one may combine the above conditions or add more conditions.

Flow Chart

[0179] Overall operation of the image process based, dynamically adjusting vehicle surveillance system for intersection traffic of Example 2 may be summarized using the summary of Example 1, FIG. 6 and Table 1, by interchanging the following names and words: PA to PC, PB to PD, right to left, and left to right.

EXAMPLE 3

[0180] Example 3 is explained using FIGS. 7, 10, and 8.

[0181] This example relates to scenario A3—referring to a view in the left-side mirror during left turn. More specifically, an image process based, dynamically adjusting vehicle surveillance system for intersection traffic of Example 3 alleviates the shortcomings of the left-side mirror during left turns as explained earlier.

[0182] Here the key desired view is a view of a left-side mirror when the vehicle is not turning left at an intersection, and the desired view is a view of a road section toward behind the vehicle on the left side before a turn is initiated when the vehicle is turning left at an intersection.

[0183] FIG. 7 gives a block diagram of the image process based, dynamically adjusting vehicle surveillance system for intersection traffic of Example 3. The image process based, dynamically adjusting vehicle surveillance system for intersection traffic of Example 3 includes the camera 10, the controller and memory unit 20, the monitor 30, and a side information source 35.

[0184] The connectivity of the camera 10, the controller and memory unit 20 and the monitor 30 are as described previously. In Example 3, the side information source 35 is connected to the controller and memory unit 30. In addition, the side information source 35 is a left side signal switch.

[0185] The camera 10 captures a sequence of views: view(1), view(2) and so on. The controller and memory unit 20 receives the sequence of the views: view(1), view(2) and so on.

[0186] When the vehicle is not making a left turn at an intersection, the controller and memory unit 20 selects portions P1, P2, and so on, of the views, view(1), view(2), and so on, respectively, such that Pi's are views of a left-side mirror. Then, the controller and memory unit 20 displays portions P1, P2, and so on, on the monitor 30. The monitor 30 provides these views to the driver of the vehicle.

[0187] When the vehicle is making a left turn at an intersection, the controller and memory unit 20 selects portions P1, P2, and so on, of the views, view(1), view(2), and so on, respectively, such that Pi's are views of a road section toward behind the vehicle on the left side before a turn is initiated.

[0188] Next the controller and memory unit 20 displays portions P1, P2, and so on, on the monitor 30. The monitor 30 provides these views for the driver of the vehicle.

[0189] How Pi potions are selected will be explained below.

Angular Position Changes

[0190] The controller and memory unit 20 is configured to measure the changes in angular position of the vehicle as in Example 1.

[0191] How a left turn is detected will be explained below.

[0192] The controller and memory unit 20 is configured to detect the following condition.

[0193] 1) A left turn signal.

How to Select Portions (P1, P2, and so on)

[0194] When a left turn is not detected, more specifically, the controller and memory unit 20 selects P1=PA, P2=PA, and so on, where PA is the predetermined area in the view of the camera 10 that corresponds to a view of a typical left-side mirror.

[0195] When a left turn is detected, more specifically, the controller and memory unit 20 initially keeps P1=PA. After receiving view(2), the controller and memory unit 20 measures the change in angular position of the vehicle from position in view(1) to position in view(2). The change in angular position can be denoted by ph2. Next, the controller and memory unit 20 defines P2=SHIFT(P1,ph2), that is the portion P2 is P1 shifted by ph2 pixels. If ph2<0, then the shift is to the left, but if ph2>0, then the shift is to the right.

[0196] Then, the controller and memory unit 20 repeats the procedure, for the next view.

[0197] Therefore, the monitor 30 provides a driver of the vehicle with the desired key views, both while making left turns and while not making left turns. In addition, while making a left turn, the monitor 30 provides a view that is generally in a consistent direction throughout most of the turn.

[0198] FIG. 10 depicts an application of the image process based, dynamically adjusting vehicle surveillance system for intersection traffic of Example 3. The trailer-truck 8 is depicted in three positions: facing north at the intersection 5 (position 22), facing north-west in the intersection 5 (position 23), and facing west past the intersection 5 (position 24).

[0199] During most of the left turn, the left-side mirror 2 shows the body of the trailer-truck 8 on the left side. Instead the monitor 30 shows a view of the road section toward behind the vehicle on the left side before a turn is initiated. In FIG. 10, there is an automobile 9 on the left side of the trailer-truck 8. The automobile 9 is in danger because it is not in the view of the driver of the trailer-truck 8. However, the automobile 9 is in the view of the monitor 30, since most of the view of the dynamically adjusting vehicle surveillance system for intersection traffic of Example 3 is toward south in a generally consistent manner.

How to Terminate a Left Turn

[0200] The controller and memory unit 20 ends a left turn and starts displaying a left-side mirror view on the monitor 30 when one of the following conditions is detected: [0201] 1. If the magnitude of total changes in the angular position of the vehicle exceeds alpha5, where alpha5 is a predetermined value, for example 45 degrees. [0202] 2. If the left turn signal is turned off.

[0203] After detecting any of the above conditions, the controller and memory unit 20 moves the portions to position PA again.

Flow Chart

[0204] Overall operation of the image process based, dynamically adjusting vehicle surveillance system for intersection traffic of Example 3 may be summarized using FIG. 8 and Table 2.

[0205] FIG. 8 differs from FIG. 6 in that instead of the boxes 61 and 62, FIG. 8 has a box 65 named “side information detector”.

[0206] Outputs: ph, tph, tNph, and tMph are generated exactly the same way as in FIG. 6, where ph is the change in angular position in one step, tph is the total change in angular position, tpNph is the total change in angular position in the last N step, and tpMph is the total change in angular position in the last M step.

[0207] The output of the box 65 named “side information detector” is a binary signal yes/no (or 0/1). In Example 3, the side information comes from the left side signal switch. If the switch is on, the output of the box 65 is yes, but if the switch is off, then the output of the box 65 is no.

[0208] The rest of the operation of the image process based, dynamically adjusting, intersection traffic, vehicle surveillance system of Example 3 may be described using Table 2.

[0209] Table 2 gives a finite state transition machine, FSTM, with two states: 1, 2.

[0210] If the state=1, it means the FSTM is processing a “not making a left turn” by the vehicle.

[0211] If the state=2, it means the FSTM, is processing a “making a left turn” by the vehicle.

[0212] For a given row, the column 2 gives the current state of the FSTM.

[0213] For a given row, the column 3 is yes if one of the following 2 conditions is satisfied, otherwise it is a no.

[0214] Condition 1: Referring to FIG. 8, the box 65 output is no.

[0215] Condition 2: The abs(tMph) of FIG. 8 is greater than alpha5.

[0216] For a given row, the column 4 is yes if the following condition is satisfied, otherwise it is a no.

[0217] Condition 1: Referring to FIG. 8, the box 65 output is yes.

[0218] For a given row, the column 5 gives the next state of the FSTM.

[0219] For a given row, the column 6 gives the next selected portion.

TABLE-US-00002 TABLE 2 1 2 3 4 5 6 1 1 N 1 PA 2 1 Y 2 PA 3 2 N 2 SHIFT(PX, ph) 4 2 Y 1 PA

[0220] The rows of Table 2 can be described as follows.

[0221] Row 1: If the current state is 1, then column 3 is not relevant. If the column 4 is no (no left turn signal), then the next state is 1 (column 5) and the next selected region is PA (corresponds to left side mirror view, column 6).

[0222] Row 2: If the current state is 1, then column 3 is not relevant. If the column 4 is yes (left turn signal detected), then the next state is 2 (column 5) and the next selected region is PA (column 6).

[0223] Row 3: If the current state is 2, then column 4 is not relevant. If the column 3 is no, then the next state is 2 (column 5) and the next selected region is PX=SHIFT(PX,ph) (column 6).

[0224] Row 4: If the current state is 2, then column 4 is not relevant. If the column 3 is yes, then the next state is 1 (column 5) and the next selected region is PX=PA (column 6).

EXAMPLE 4

[0225] Example 4 is explained using FIGS. 7, 11, and 8.

[0226] This example relates to scenario A2—referring to a view in the right-side mirror during right turn. More specifically, an image process based, dynamically adjusting vehicle surveillance system for intersection traffic of Example 4 alleviates the shortcomings of the right-side mirror during right turns as explained earlier.

[0227] Here the key desired view is a view of a right-side mirror when the vehicle is not turning right at an intersection, and the desired view is a view of a road section toward behind the vehicle on the right side before a turn is initiated when the vehicle is turning right at an intersection.

[0228] An image process based, image process based, dynamically adjusting vehicle surveillance system for intersection traffic of example 4 comprises the camera 10, the controller and memory unit 20, the monitor 30, and the side information source 35 of FIG. 7.

[0229] In Example 4, the side information source 35 is a GPS unit.

[0230] The camera 10 captures a sequence of views: view(1), view(2), and so on. The controller and memory unit 20 receives the sequence of the views: view(1), view(2) and so on.

[0231] When the vehicle is not making a right turn at an intersection, the controller and memory unit 20 selects portions P1, P2, and so on, of the views, view(1), view(2), and so on, respectively, such that Pi's are views of a right-side mirror, next the controller and memory unit 20 displays portions P1, P2, and so on, on the monitor 30. The monitor 30 provides these views to the driver of the vehicle.

[0232] When the vehicle is making a right turn at an intersection, the controller and memory unit 20 selects portions P1, P2, and so on, of the views, view(1), view(2), and so on, respectively, such that Pi's are views of a road section toward behind the vehicle on the right side before a turn is initiated.

[0233] Next the controller and memory unit 20 displays portions P1, P2, and so on, on the monitor 30. The monitor 30 provides these views for the driver of the vehicle.

[0234] How a right turn is detected will be explained below.

[0235] The controller and memory unit 20 is configured to detect the following condition.

[0236] 1) GPS signals indicating proximity to the intersection and direction of turn (right turn for example 4).

How to Select Portions (P1, P2, and so on)

[0237] When a right turn is not detected, more specifically, the controller and memory unit 20 selects P1=PC, P2=PC, so on, where PC is the predetermined area in the view of the camera 10 that corresponds to a view of a typical right-side mirror.

[0238] When a right turn is detected, more specifically, the controller and memory unit 20 initially keeps P1=PC. After receiving, view(2), the controller and memory unit 20 measures the change in angular position of the vehicle from position in view(1) to position in view(2). The change of angular position can be denoted by ph2. Next, the controller and memory unit 20 defines P2=SHIFT(P1,ph2), that is the portion P2 is P1 shifted by ph2 pixels. If ph2<0, then the shift is to the left, but if ph2>0, then the shift is to the right.

[0239] Then, the controller and memory unit 20 repeats the procedure, for the next view.

[0240] Therefore, the monitor 30 provides a driver of the vehicle with the desired key views, both while making right turns and while not making right turns. In addition, while making a right turn, the monitor 30 provides a view that is generally in a consistent direction throughout most of the turn.

[0241] FIG. 11 depicts an application of the image process based, dynamically adjusting vehicle surveillance system for intersection traffic of Example 4. The trailer-truck 8 is depicted in three positions: facing north at the intersection 5 (position 22), facing north-east in the intersection 5 (position 25), and facing east past the intersection 5 (position 26).

[0242] During most of the right turn, the right-side mirror 1 shows the body of the trailer-truck 8 on the right side. Instead, the monitor 30 shows a view of the road section toward behind the vehicle on the right side before a turn is initiated. In FIG. 11, there is the automobile 9 on the right side of the trailer-truck 8. The automobile 9 is in danger because it is not in the view of the driver of the trailer-truck 8. However, the automobile 9 is in the view of the monitor 30, since most of the view of the dynamically adjusting vehicle surveillance system for intersection traffic of Example 4 is toward south in a generally consistent manner.

How to Terminate a Right Turn

[0243] The controller and memory unit 20 ends a right turn and starts displaying a right-side mirror view on the monitor 30 when one of the following conditions is detected: [0244] 1. If the magnitude of total changes in the angular position of the vehicle exceeds alpha5, where alpha5 is a predetermined value, for example 45 degrees. [0245] 2. If a timer runs out. The duration of the timer is predetermined.

[0246] After detecting any of the above condition, the controller and memory unit 20 moves the portions to position PC again.

Flow Chart

[0247] Overall operation of the image process based, image process based, dynamically adjusting vehicle surveillance system for intersection traffic of Example 4 may be summarized using FIG. 8 and Table 3.

[0248] The description of FIG. 8 of Example 3 directly applies to Example 4, except the side information of the box 65 are GPS signals indicating proximity to the intersection and direction of turn (right turn for Example 4). If the GPS signal are received then the output of the box is ‘yes’, otherwise it is ‘no’.

[0249] The rest of the operation of the image process based, image process based, dynamically adjusting vehicle surveillance system for intersection traffic of Example 4 may be described using Table 3.

[0250] Table 3 gives a finite state transition machine, FSTM, with two states: 1, 2.

[0251] If the state=1, it means the FSTM is processing a “not making a right turn” by the vehicle.

[0252] If the state=2, it means the FSTM, is processing a “making a right turn” by the vehicle.

[0253] For a given row, the column 2 gives the current state of the FSTM.

[0254] For a given row, the column 3 is yes if one of the following 2 conditions is satisfied, otherwise it is a no.

[0255] Condition 1: A counter, con0=N0, N0 a predetermined value.

[0256] Condition 2: The abs(tMph) of FIG. 8 is greater than alpha5.

[0257] For a given row, the column 4 is yes if the following condition is satisfied, otherwise it is a no.

[0258] Condition 1: Referring to FIG. 8, the box 65 output is yes; detection of GPS signals indicating proximity of the vehicle to the intersection and the direction of turn (right turn for example 4).

[0259] For a given row, the column 5 gives the next state of the FSTM.

[0260] For a given row, the column 6 gives the next selected portion.

[0261] For a given row, the column 7 states whether the counter con0 should be incremented or reset to zero or put to sleep.

TABLE-US-00003 TABLE 3 1 2 3 4 5 6 7 1 1 N 1 PC Null 2 1 Y 2 PC 0 3 2 N 2 SHIFT(PX, ph) Add 1 4 2 Y 1 PC Null

[0262] The rows of Table 3 can be defined as follows.

[0263] Row 1: If the current state is 1, then column 3 is not relevant. If the column 4 is no (no GPS signals for proximity and direction of turn), then the next state is 1 (column 5) and the next selected region is PC (corresponds to right side mirror view, column 6). The counter con0 is in sleep mode (column 7).

[0264] Row 2: If the current state is 1, then column 3 is not relevant. If the column 4 is yes (GPS signals for proximity and direction of turn), then the next state is 2 (column 5) and the next selected region is PC (column 6). The counter con0 is reset to zero (column 7).

[0265] Row 3: If the current state is 2, then column 4 is not relevant. If the column 3 is no, then the next state is 2 (column 5) and the next selected region is PX=SHIFT(PX,ph) (column 6), and counter con0 is incremented (column 7).

[0266] Row 4: If the current state is 2, then column 4 is not relevant. If the column 3 is yes, then the next state is 1 (column 5) and the next selected region is PX=PC (column 6), and the counter con0 is put to sleep (column 7).

[0267] When two image process based, dynamically adjusting vehicle surveillance systems for intersection traffic are used, one for the right side and one for the left side of the automobile, then one may use the views from both sides in the computation of the changes of angular position. That is one may find a rotation that would produce an overall best (on both sides) reproduction to Q1.

[0268] The region Q1 does not have to be a connected region, that is, one may use a region Q1 that is a union of two disjoint regions of the view(1).

[0269] Examples 1 through 4 apply to automobiles as well as trucks, trailer-trucks and other like vehicles.

[0270] The controller and memory unit 20 may do additional image processing to enhance the images it sends to the monitor 30. For example, the controller and memory unit 20 may automatically adjust for brightness and remove vibration (stabilize). The parameter, pv, related to the regions Q1 and Q2, is especially useful to stabilization algorithms.

[0271] For very wide angle view of a side of the vehicle, more than one camera may be used on each side. The view would then be a panorama view of the views of the cameras.

[0272] All predetermined parameters may be programmable and changeable to serve different users preferences.

[0273] Example 3 has a favorable use during lane changes. During lane changes the side mirrors have the same disadvantages they have during turns, but milder. Still, the image process based, dynamically adjusting vehicle surveillance systems for intersection traffic produces more desired and useful views then traditional side mirrors. The key view during a lane change is the view of the traffic in the lane to which a vehicle is moving. As a vehicle is changing lanes, the side mirror view gets weaker with respect to the key view, whereas in the systems of Example 3, the view of the monitor gets stronger with respect to the key view, that is it spans more of the key view.

[0274] If GPS is used in Examples 1 and 2, the controller and memory unit 20 may be further configured to incorporate the GPS signals. More specifically, the controller and memory unit 20 may update the position of the selected subsets of the views, P's, based on side information from the GPS. More specifically, PB and PD are adjusted based on the information on the number of lanes, the angle of intersection between the intersecting roads at the intersection, and the like. Also, the controller and memory unit 20 may terminate the dynamically adjusting surveillance system based on information from the GPS.

[0275] One may use an image process based, dynamically adjusting vehicle surveillance system to measure the changes in the angular position of the camera 10. To this end, one may use the camera 10 and the controller and memory unit 20, and further use the controller and memory unit 20 to measure the change in the angular position of the camera based on the sequence of views view(1), view(2), and so on.

[0276] The dynamically adjusting vehicle surveillance systems may choose, instead of sending the views to a display or the monitor 30, to record them for later analysis.

[0277] Many alterations and modifications may be made by those having ordinary skill in the art without departing from the spirit and scope of the invention. Therefore, it must be understood that the illustrated embodiments have been set forth only for the purposes of examples and that they should not be taken as limiting the invention as defined by the following claims. For example, notwithstanding the fact that the elements of a claim are set forth below in a certain combination, it must be expressly understood that the invention includes other combinations of fewer, more or different ones of the disclosed elements.

[0278] The words used in this specification to describe the invention and its various embodiments are to be understood not only in the sense of their commonly defined meanings, but to include by special definition in this specification the generic structure, material or acts of which they represent a single species.

[0279] The definitions of the words or elements of the following claims are, therefore, defined in this specification to not only include the combination of elements which are literally set forth. In this sense it is therefore contemplated that an equivalent substitution of two or more elements may be made for any one of the elements in the claims below or that a single element may be substituted for two or more elements in a claim. Although elements may be described above as acting in certain combinations and even initially claimed as such, it is to be expressly understood that one or more elements from a claimed combination can in some cases be excised from the combination and that the claimed combination may be directed to a subcombination or variation of a subcombination.

[0280] The claims are thus to be understood to include what is specifically illustrated and described above, what is conceptually equivalent, what can be obviously substituted and also what incorporates the essential idea of the invention.