DETECTION OF MOVING OBJECTS

Abstract

A camera (10) for detecting objects (48) moving relative to the camera (10) in a direction of movement (50), comprising an image sensor (18) for recording image data of the objects (48) in a camera field of view (14, 56), an optoelectronic distance sensor (24) using a time-of-flight method having a plurality of measurement zones (30a) for measuring a plurality of distance values to the objects (48) in a distance measurement field of view (58), and a control and evaluation unit (38) configured to find, by measuring distance values over a configuration time and evaluating the distance values and/or their change, a region where objects (48) move, and to automatically set a region of interest (60) for the distance sensor (24) within the distance measurement field of view (58) by determining an object region as the region where objects (48) move.

Claims

1. A camera (10) for detecting objects (48) moving relative to the camera (10) in a direction of movement (50), the camera (10) comprising an image sensor (18) for recording image data of the objects (48) in a camera field of view (14, 56), an optoelectronic distance sensor (24) according to a principle of a time-of-flight method having a plurality of measurement zones (30a) for measuring a plurality of distance values to the objects (48) in a distance measurement field of view (58), and a control and evaluation unit (38) connected to the image sensor (18) and the distance sensor (24), the control and evaluation unit (38) being configured to find, by measuring distance values over a configuration time and evaluating at least one of the distance values and a change of the distance values, a region where objects (48) move, and to automatically set a region of interest (60) for the distance sensor (24) within the distance measurement field of view (58) by determining an object region as the region where objects (48) move.

2. The camera (10) according to claim 1, wherein the distance sensor (24) is integrated into the camera (10).

3. The camera (10) according to claim 1, wherein the measuring zones (30a) can be read out in parallel.

4. The camera (10) according to claim 1, wherein the control and evaluation unit (38) is configured to determine a statistical measure for changes in the distance values of a respective measurement zone (30a) over a variation time.

5. The camera (10) according to claim 1, wherein the control and evaluation unit (38) is configured to determine a respective maximum value of the distance values of a respective measurement zone (30a) over a background detection time.

6. The camera (10) according to claim 5, wherein the control and evaluation unit (38) is configured to adjust the region of interest (60) based on the maximum values and an expected background geometry.

7. The camera (10) according to claim 6, wherein the adjustment concerns a direction transverse to the direction of movement (50).

8. The camera (10) according to claim 1, wherein the control and evaluation unit (38) is configured to restrict the region of interest (60) along the direction of movement (50) to an entrance region directed towards the moving objects (48).

9. The camera (10) according to claim 1, wherein the distance sensor (24) comprises a plurality of avalanche photodiodes (30a) operable in Geiger mode, and wherein the control and evaluation unit (38) is configured to adjust the region of interest (60) by selectively activating the Geiger mode in avalanche photodiodes (30a) corresponding to the region of interest (60).

10. The camera (10) according to claim 1, wherein the distance measurement field of view (58) projects beyond the camera field of view (56).

11. The camera (10) according to claim 10, wherein the distance measurement field of view (58) projects beyond the camera field of view (56) against the direction of movement (50).

12. The camera (10) according to claim 1, wherein the control and evaluation unit (38) is configured to read out code contents of codes (52) in the recorded image data of the objects (48).

13. The camera (10) according to claim 1, which is stationarily mounted on a conveyor device (46) which conveys the objects (48) in the direction of movement (50).

14. A method for detecting objects (48) moving in a direction of movement (50), wherein image data of the objects (48) are recorded by a camera (10) in a camera field of view (56) and a plurality of distance values to the objects (48) are measured with an optoelectronic distance sensor (24) according to a principle of a time-of-flight method with a plurality of measurement zones (30a) in a distance measurement field of view (58), wherein image data are read out and distance values are evaluated, wherein an region of interest (60) for the distance sensor (24) within the distance measurement field of view (58) is automatically configured by measuring distance values over a configuration time and evaluating at least one of the distance values and a change of the distance values in order to determine an object region as the region where objects (48) move.

Description

[0028] The invention will be explained in the following also with respect to further advantages and features with reference to exemplary embodiments and the enclosed drawing. The Figures of the drawing show in:

[0029] FIG. 1 a schematic sectional view of a camera having an optoelectronic distance sensor;

[0030] FIG. 2 a three-dimensional view of an exemplary application of the camera mounted at a conveyor belt;

[0031] FIG. 3 a schematic sectional view of the camera and the respective fields of view of the camera and distance sensor;

[0032] FIG. 4 an exemplary illustration of a camera having an adapted field of view of the distance sensor above a conveyor belt with lateral structures; and

[0033] FIG. 5 a representation of the measured distance values in dependence on the lateral position of a conveyor belt as well as the adjusted field of view of the distance sensor in the example situation of FIG. 4.

[0034] FIG. 1 shows a schematic sectional view of a camera 10. Received light 12 from a detection area 14 impinges on a receiving optics 16, which guides the received light 12 to an image sensor 18. The optical elements of the receiving optics 16 are preferably designed as an objective comprising a plurality of lenses and other optical elements such as diaphragms, prisms and the like, but are merely represented as a single lens for simplification.

[0035] In order to illuminate the detection area 14 with transmitted light 20 during image acquisition of the camera 10, the camera 10 comprises an optional illumination unit 22, which is shown in FIG. 1 in the simplified form of a light source without transmission optics. In other embodiments, multiple light sources, such as LEDs or laser diodes, are arranged, for example, in a ring around the receiving path, which can also be multicolored and controllable in groups or individually to adjust parameters of the illumination unit 22 such as its color, intensity and direction.

[0036] In addition to the actual image sensor 18 for capturing image data, the camera 10 has an optoelectronic distance sensor 24 that uses a time-of-flight (ToF) method to measure distances to objects in the detection area 14. The distance sensor 24 includes a TOF light transmitter 26 with TOF transmitter optics 28 and a TOF light receiver 30 with TOF receiver optics 32, for transmitting and receiving a TOF light signal 34. A light time-of-flight measuring unit 36 determines the time of flight of the TOF light signal 34 and, based thereon, the distance to an object where the TOF light signal 34 was reflected.

[0037] The TOF light receiver 30 comprises a plurality of light receiving elements 30a. The light receiving elements 30a, individually or in smaller groups, form measuring zones, each determining a distance value. Therefore, not only a single distance value is detected, but the distance values are spatially resolved and can be combined to form a height profile. The measuring zones can preferably be read out in parallel so that a consistent height profile is generated when there is a relative movement between the camera 10 and the detected object. The number of measurement zones of the TOF light receiver 30 may remain comparatively small, for example a few tens, hundreds or thousands of measurement zones, far from usual megapixel resolutions of the image sensor 18.

[0038] The shown configuration of the distance sensor 24 is just an example. Optoelectronic distance measurement using time-of-flight light methods is known and therefore will not be explained in detail. Two exemplary measurement methods are photonic mixing detection (PMD) using a periodically modulated TOF light signal 34 or pulse time-of-flight measurement using a pulse-modulated TOF light signal 34. Highly integrated solutions exist, with the TOF light receiver 30 arranged on a common chip together with the time-of-flight measurement unit 36 or at least parts thereof, such as TDCs (time-to-digital converters) for time-of-flight measurements. A TOF light receiver 30 that is configured as a matrix of SPAD light receiving elements 30a (Single-Photon Avalanche Diode) is particularly suitable for this purpose. Measurement zones of SPAD light-receiving elements 30a can be selectively deactivated and activated by setting a bias voltage below or above the breakdown voltage. This allows an active region of the distance sensor 24 to be set. TOF optics 28, 32 are shown as a respective symbol of a single lens only, representing any optics such as a microlens array.

[0039] A control and evaluation unit 38 is connected to the illumination unit 22, the image sensor 18 and the distance sensor 38, and it is responsible for the control, evaluation and other coordination tasks in the camera 10. It thus reads image data from the image sensor 18 in order to store or output that image data at an interface 40. Depending on the embodiment, the control and evaluation unit 38 uses the distance values of the distance sensor 24 for various purposes, for example to determine or set camera parameters, to trigger camera functions or to evaluate image data, including preprocessing for an actual evaluation in the camera 10 or a higher-level system. Preferably, the control and evaluation unit 38 is capable of locating and decoding code regions in the image data, making the camera 10 a camera-based code reader.

[0040] The camera 10 is protected by a housing 42, having a front window 44 in the front area where the received light 12 is incident.

[0041] FIG. 2 shows a possible application of the camera 10 stationarily mounted at a conveyor belt 46. Here and in the following, the camera 10 is shown only as a single symbol and no longer with its structure that already has been explained with reference to FIG. 1. The conveyor belt 46 conveys objects 48, as indicated by a direction of movement 50 with an arrow, through the detection area 14 of the camera 10. The objects 48 may bear code areas 52 on their outer surfaces. The task of the camera 10 is to detect properties of the objects 48 and, in a preferred application as a code reader, to detect the code areas 52, to read out the codes, to decode the code content, and to assign the code and its code content to the respective associated object 48. In order to also detect object sides and in particular lateral code areas 54 of codes attached to side surfaces, additional cameras 10, not shown, are preferably used from various perspectives.

[0042] FIG. 3 again shows a camera 10 in a sectional view to explain the different fields of view (FOV). In the detection area 14, a camera field of view 56 of the actual camera 10 or its image sensor 18 shown with dotted lines and a distance measurement field of view 58 of the distance sensor 24 shown with dashed lines are distinguished.

[0043] According to the invention, the camera 10 is capable of automatically setting a region of interest 60 for the distance sensor 24 as a partial region of the distance measurement field of view 58. For this purpose, it is advantageous if the distance measurement field of view 58 is larger than the camera field of view 56, at least in a direction against the direction of movement 50 corresponding to the incoming objects 48. In order not to limit the possible mounting directions of the camera 10, the distance measurement field of view 58 may be larger in all directions.

[0044] The distance sensor 24 detects objects 48 in the region of interest 60 in good time so that the camera 10 has time to adjust to the distance values before the object 48 has reached an acquisition position, for example, in the center of the camera field of view 56. In contrast, an area 62 facing away from the acquisition position is no longer of interest for the distance measurement, at least not as far as settings of the camera 10 are concerned, since such measurements are too late. Depending on the acquisition position, the measuring frequency of the distance sensor 24, and the required duration for distance-dependent adjustments of the camera 10, the area 62 can start earlier, i.e. be shifted farther to the left in FIG. 3. One might call the area 62 the past and the area to the left thereof the future, since objects 48 have already left the acquisition position or will reach it, respectively.

[0045] Accordingly, a very coarse selection of the region of interest 60 might only exclude the region 62. However, the region of interest 60 preferably is set more precisely. On the one hand, this may be done along the direction of movement 50 by restriction to an earliest entry range of the objects 48, as shown in FIG. 3. In addition, an adjustment of the region of interest 60 can be made transverse to the direction of movement 50. The goal is to both cover the conveyor belt 46 completely and not to cover any side regions beyond the conveyor belt 46. A precisely set region of interest 60 reduces the amount of data and required bandwidth and thus also simplifies the evaluation. In addition, it is conceivable to increase the measuring frequency of the distance sensor 24 if its operating range is reduced to fewer measuring zones. As mentioned above, the region of interest 60 is preferably adjusted by selectively activating corresponding SPAD light receiving elements 30a in the associated measurement zones.

[0046] An exemplary automatic method for setting the region of interest 60 will now be explained. In principle, the region of interest 60 could be set manually, for example in a configuration menu of the camera 10. According to the invention, however, this should be done automatically, for example by the camera 10 starting a corresponding self -calibration at the push of a button, automatically setting and activating the appropriate region of interest 60 for the distance sensor 24.

[0047] The adjustment of the region of interest 60 relates to the direction along and across the direction of movement 50. Along the direction of movement 50, as explained with respect to FIG. 3, a distinction is to be made between past and future. This is done on the basis of the direction of movement 50, for example, by activating those measuring zones which are oriented furthest towards the incoming objects 48, such as a first column of light receiving elements 30a. Parameters of the movement of the objects 48, such as the direction of movement 50 and also the speed v of the conveyor belt 46, can be parameterized, be communicated by a connected higher-level controller, or be measured. For an autonomous measurement of these parameters, a height profile of at least one object 48 may be tracked, for example by detecting distance values associated with a time stamp.

[0048] The measurement repetition frequency of the distance sensor 24 should be high enough to detect an object 48 once or preferably a plurality of times before it has moved to the acquisition position. Formally, this may be expressed as the condition Δt.sub.TOF<<D/v, with measurement period Δt.sub.TOF of the distance sensor 24, distance D between the entry position of the objects 48 into the region of interest 60 and the acquisition position, and speed v of the conveyor belt 46. In the situation of FIG. 3, the distance D can be calculated via the half extent of the distance measurement field of view 58 at the level of the conveyor belt 46.

[0049] In the direction transverse to the direction of movement 50, the region of interest 60 preferably is set just so that all objects 48 on the conveyor belt 46 and no other objects are detected. One possible criterion is that just the width of the conveyor belt 46 should be encompassed. In an automatic configuration method, the object flow may be observed for a certain period of time. This can be done during a configuration phase prior to operation, but it is also possible to start with a region of interest 60 that is poorly adjusted, or not adjusted at all, with an optimal region of interest being found during ongoing operation.

[0050] Based on this observation and continuous measurement of distance values, on the one hand, the background can be determined, namely the conveyor belt 46 and any lateral areas next to it that are also within the distance measurement field of view 58. To that end, the respective maximum distance value is determined for each measurement zone. The objects 48 are regarded as transient, the distance sensor 24 may be conceived as waiting, by means of the maximum value determination, for a suitable moment per measuring zone when the background at least briefly becomes visible.

[0051] On the other hand, the change in the distance values per measurement zone is determined, for example in the form of a variance or standard deviation. There is a significant change only in the measuring zones through which an object 48 has moved. In the other measurement zones, only the same distance is always observed. In order to exclude a change caused by noise, a noise threshold preferably is defined. The measuring zones showing more than noise-related change are selected as the object region.

[0052] The region of interest 60 can be identified with the object region. Preferably, however, some further adjustment is made. There may be regions that do belong to the conveyor belt 46 but where by chance no object 48 has ever moved during the time of observation. Since it is known that the conveyor belt 46 forms a plane, the object region can be extended to where the same plane is continued on the basis of the background.

[0053] Conversely, high objects 48 in particular may have been detected by measuring zones that are not directed at the conveyor belt 46. This is illustrated in FIG. 4. Here, the conveyor belt 46 is framed by lateral structures 64, such as poles or similar interfering objects or structures. A measuring zone with measuring beam 66 has detected a high object 48 and has therefore initially been assigned to the object region. Without the object 48, however, the measuring beam 66 does not actually impinge on the conveyor belt 46, but on the lateral structures 64.

[0054] As illustrated in FIG. 5, this case can be detected by the background measurement, and the corresponding measuring zone of the measuring beam 66 can be excluded from the region of interest 60, although it was initially part of the object region. Shown is the distance value as a function of the position on the conveyor belt in a direction transverse to the direction of movement 50. The flat section in the center corresponds to the conveyor belt 46, while laterally the distance values deviate. The latter is done in the example of FIG. 5 with a flatter profile than in FIG. 4. Measurement zones where the maximum distance value of the background measurement does not correspond to the flat conveyor belt 46 can thus be excluded. Preferably, a tolerance threshold is taken into account, which depends on the specific environment and is preferably determined for each application.