REDUCTION OF VIDEO MATERIAL TO MOTION SECTIONS

Abstract

Systems and methods are provided that includes the processing of video material for reducing video material to temporal segments in which a significant movement of an object is recorded. The systems and methods may be used for the observation of animals.

Claims

1. A method for reducing video recordings to temporal segments in which one or more movements of an object above a motion threshold are recorded, the method including steps comprising: a) receiving a temporal sequence of images, b) generating a sequence of difference images from the sequence of images, by generating a respective difference image for each pair of adjacent images of the temporal sequence of images, c) generating a sequence of average-value difference images from the sequence of difference images, by a procedure in which, for all groups of successive difference images having a defined number of difference images, the difference images associated with a group are averaged in each case, d) generating a sequence of binary images from the sequence of average-value difference images, e) identifying groups of contiguous pixels in each binary image, f) determining the sizes of the groups of contiguous pixels and comparing the respective size of a group with a threshold value, g) identifying those binary images which have at least one group which is of a size equal to the threshold value or which is greater than the threshold value, and h) erasing all images of the temporal sequence of images which have not influenced the generation of a binary image identified in step g).

2. The method as claimed in claim 1, wherein each image of the sequence of images has a multiplicity of pixels, wherein each pixel is characterized by a tonal value, wherein each difference image of the sequence of difference images is characterized by a multiplicity of pixels each having a tonal value, wherein the tonal value of each pixel of each difference image represents an absolute value of the difference between the tonal values of the corresponding pixels of two temporally directly successive images, wherein a time window is defined for generating the sequence of average-value difference images, which time window can accommodate a defined number of temporally directly successive difference images, wherein the time window is shifted image by image from the beginning of the sequence of difference images until the end of the sequence of difference images, and, during each shifting image by image, a respective average-value difference image is generated on the basis of the difference images encompassed by the time window, wherein the tonal value of each pixel of each average-value difference image represents an average value of the tonal values of the corresponding pixels of the difference images encompassed by the time window, wherein, for generating the sequence of binary images from the sequence of average-value difference images, the tonal values of all pixels of each average-value difference image which lie below a defined tonal-value threshold value are set to a first tonal value, and the tonal values of all pixels of each (contrast-reduced) average-value difference image which lie above the defined tonal-value threshold value or correspond to the defined tonal-value threshold value are set to a second tonal value.

3. The method as claimed in claim 1, furthermore comprising the following step after step c) and before step d): generating a sequence of contrast-reduced average-value difference images by applying a gaussian blur to all average-value difference images of the sequence of average-value difference images.

4. The method as claimed in claim 1, furthermore comprising the following step after step d) and before step e): generating a sequence of dilated binary images from the sequence of binary images, by a procedure in which those pixels of each binary image which have the second tonal value are expanded singly or multiply to a shape of a defined structuring element.

5. The method as claimed in claim 1, wherein the size of each group in step f) is set as the number of pixels having a second tonal value which belong to the respective group.

6. The method as claimed in claim 1, wherein, for ascertaining the size of a group of contiguous pixels in a binary image, a bounding border around the group is considered which satisfies the following criteria: the bounding border is rectangular, its edges run parallel to the edges of the binary image, all pixels which have the second tonal value and which belong to a group of contiguous pixels lie within the bounding border, the bounding border comprises as few pixels as possible which do not belong to the group of contiguous pixels having the second tonal value, wherein the total number of pixels lying within the bounding border is set as the size of the group.

7. The method as claimed in claim 1, wherein the object is a living organism, preferably a living organism in the form of an animal.

8. The method as claimed in claim 7, wherein the at least one movement is shaking of the body of the living organism or of part of the body of the living organism, activities of the living organism concerning its own body care, and/or licking, chewing, scratching and/or rubbing of the living organism.

9. A device comprising: an input unit, a control unit, a computing unit, and an output unit and/or a data storage unit, wherein the control unit is configured to cause the input unit to receive a sequence of images, wherein the control unit is configured to cause the computing unit to carry out the following steps: a) generating a sequence of difference images from the sequence of images, by generating a respective difference image for each pair of adjacent images of the temporal sequence of images, b) generating a sequence of average-value difference images from the sequence of difference images, by a procedure in which, for all groups of successive difference images having a defined number of difference images, the difference images associated with a group are averaged in each case, c) generating a sequence of binary images from the sequence of average-value difference images, d) identifying groups of contiguous pixels in each binary image, e) determining the sizes of the groups of contiguous pixels and comparing the respective size with a threshold value, f) identifying those binary images which have at least one group which is of a size equal to the threshold value or which is greater than the threshold value, g) erasing all images of the temporal sequence of images which have not influenced the generation of a binary image identified in step f), wherein a reduced sequence of images is generated, wherein the control unit is configured to store the reduced sequence of images in the data storage unit and/or to cause the output unit to output the reduced sequence of images.

10. The device as claimed in claim 9, wherein the control unit is configured to cause the computing unit to: a) receive a temporal sequence of images, b) generate a sequence of difference images from the sequence of images, by generating a respective difference image for each pair of adjacent images of the temporal sequence of images, c) generate a sequence of average-value difference images from the sequence of difference images, by a procedure in which, for all groups of successive difference images having a defined number of difference images, the difference images associated with a group are averaged in each case, d) generate a sequence of binary images from the sequence of average-value difference images, e) identify groups of contiguous pixels in each binary image, f) determine the sizes of the groups of contiguous pixels and comparing the respective size of a group with a threshold value, g) identify those binary images which have at least one group which is of a size equal to the threshold value or which is greater than the threshold value, and h) erase all images of the temporal sequence of images which have not influenced the generation of a binary image identified in step g).

11. The device as claimed in claim 10, wherein the object is a living organism, preferably a living organism in the form of an animal.

12. A computer program product comprising a non-transitory computer storage medium having a computer program stored thereon that causes a computer to: a) receive a temporal sequence of images, b) generate a sequence of difference images from the sequence of images, by generating a respective difference image for each pair of adjacent images of the temporal sequence of images, c) generate a sequence of average-value difference images from the sequence of difference images, by a procedure in which, for all groups of successive difference images having a defined number of difference images, the difference images associated with a group are averaged in each case, d) generate a sequence of binary images from the sequence of average-value difference images, e) identify groups of contiguous pixels in each binary image, f) determine the sizes of the groups of contiguous pixels and comparing the respective size of a group with a threshold value, g) identify those binary images which have at least one group which is of a size equal to the threshold value or which is greater than the threshold value, h) erase all images of the temporal sequence of images which have not influenced the generation of a binary image identified in step g), and thereby generating a reduced sequence of images, and i) store the reduced sequence of images in a data storage unit and/or outputting the reduced sequence of images on a monitor.

Description

[0040] In one preferred embodiment, in a next step for suppressing noise in the average-value difference images the contrast of the average-value difference images is reduced. The contrast is preferably reduced by blur, e.g. by applying a Gaussian blur.

[0041] A Gaussian blur uses a Gaussian filter to smooth image contents. The filter results in a reduction of image noise and causes smaller structures to disappear in order to obtain coarser regions. A Gaussian blur acts on each pixel of an average-value difference image and sets its tonal value to a weighted average value of the tonal values of all pixels that lie in a defined radius with respect to the pixel under consideration. The weighting is effected on the basis of the Gaussian normal distribution. Gaussian blurs are known to the person skilled in the art of digital image processing (see e.g. William K. Pratt: Introduction to Digital Image Processing, CRC Press, 2013, ISBN: 978-1-4822-1670-7) and they are implemented in many image processing software programs. Parameters that have to be predefined for the application of a Gaussian blur are the standard deviation of the Gaussian function (sigma) and the size of the radius or the size of the matrix of pixels which are intended to be taken into account in the weighted averaging (kernel). Adequate parameter values can be determined empirically. Examples are parameter values are sigma=0.5 or sigma=1 and kernel=3×3 or kernel=5×5.

[0042] The result of the contrast reduction is a sequence of contrast-reduced average-value difference images.

[0043] Each contrast-reduced average-value difference image is preferably assigned the point in time of the corresponding average-value difference image from which it was generated.

[0044] If the average-value difference images have a unique identifier, then the contrast-reduced average-value difference images preferably likewise have a unique identifier. The unique identifiers of the contrast-reduced average-value difference images can comprise for example the unique identifiers of the corresponding average-value difference images from which they were generated.

[0045] A contrast-reduced average-value difference image usually has exactly the same number of pixels as the average-value difference image from which it was generated.

[0046] In a further step, the average-value difference images or the contrast-reduced average-value difference images are binarized. That means that each pixel of an average-value difference image is assigned one of two tonal values—a first tonal value or a second tonal value. The first tonal value can have the value 0 (black) for example and the second tonal value can have the value 255 (white) for example. The assignment is effected on the basis of the existing tonal value of a pixel and on the basis of a tonal-value threshold value. If the tonal value of a pixel is less than the tonal-value threshold value, then the tonal value of the pixel is set to the first tonal value; if the tonal value of the pixel is greater than the tonal-value threshold value or the tonal value of the pixel corresponds to the tonal-value threshold value, then the tonal value of the pixel is set to the second tonal value.

[0047] An adequate tonal-value threshold value can be determined empirically. Examples of tonal-value threshold values (in the case of a tonal value range of 0 to 255) are 10 or 20 or 30 or 50.

[0048] The result of the binarization is a sequence of binarized average-value difference images, which are also referred to as binary images in this description.

[0049] Each binary image is preferably assigned a point in time. The point in time of a binary image preferably corresponds to the point in time assigned to that (contrast-reduced) average-value difference image from which the binary image was generated.

[0050] If the average-value difference images have a unique identifier, then the binary images preferably likewise have a unique identifier. The unique identifiers of the binary images can comprise for example the unique identifiers of the corresponding (contrast-reduced) average-value difference images.

[0051] A binary image usually has exactly the same number of pixels as the (contrast-reduced) average-value difference image from which it was generated.

[0052] In one preferred embodiment, in a further step, those pixels of the binary images which have the second tonal value are expanded (dilated) singly or multiply to the shape of a defined structuring element.

[0053] In one preferred embodiment, the structuring element is an (n×m) matrix of pixels having the second tonal value. That means that a pixel having a second tonal value is expanded to an (n×m) matrix of pixels having a second tonal value.

[0054] In one particularly preferred embodiment, a (3×3)-matrix is involved. Particularly preferably, such a dilatation operator is applied twice in succession.

[0055] The result of such a single or multiple dilatation is a sequence of dilated binary images.

[0056] Each dilated binary image is preferably assigned a point in time. The point in time of a dilated binary image preferably corresponds to the point in time assigned to that binary image from which the dilated binary image was generated.

[0057] If the binary images have a unique identifier, then the dilated binary images preferably likewise have a unique identifier. The unique identifiers of the dilated binary images can comprise for example the unique identifiers of the corresponding binary images.

[0058] A dilated binary image usually has exactly the same number of pixels as the binary image from which it was generated.

[0059] In a next step, in each (dilated) binary image of the sequence of binary images, groups of contiguous pixels which have the second tonal value are identified and the respective sizes of these groups are ascertained.

[0060] In raster graphics, the pixels in the four corners of the raster graphics have in each case three directly adjacent pixels, the pixels at the edges of the raster graphics in each case have five directly adjacent pixels, and the rest of the pixels of the raster graphics have in each case eight directly adjacent pixels. Contiguous pixels having the second tonal value are all those pixels whose tonal value corresponds to the second tonal value and which have at least one directly adjacent pixel whose tonal value likewise corresponds to the second tonal value.

[0061] The size of a group can be ascertained and specified by way of the number of pixels having a second tonal value which belong to the group.

[0062] It is also conceivable for the size of a group to be ascertained and specified by way of the area taken up by this group.

[0063] In one preferred embodiment, the size of a group is ascertained by way of the number of pixels in a bounding border that bounds groups of contiguous pixels having a second tonal value.

[0064] In this preferred embodiment, groups of contiguous pixels having the second tonal value are bounded by a bounding border, wherein the bounding border is chosen such that it satisfies all the following criteria:

the bounding border is rectangular,
its edges run parallel to the edges of the binary image,
all pixels which have the same tonal value and which belong to a group of contiguous pixels lie within the bounding border,
the bounding border comprises as few pixels as possible which do not belong to the group of contiguous pixels having the second tonal value.

[0065] The sizes of the groups of contiguous pixels having the second tonal value are compared in each case with a group threshold value. If binary images have exclusively pixels having a first tonal value, no corresponding groups are present. If a binary image includes at least one group which is of a size exactly equal to the group threshold value or which is greater than the group threshold value, then the corresponding binary image indicates a movement above a motion threshold. If a binary image includes only one or a plurality of groups smaller than the group threshold value, then the corresponding binary image indicates no movement above a motion threshold. Binary images that indicate no movement above a motion threshold are of no interest for further evaluation.

[0066] Accordingly, it is necessary to identify those (dilated) binary images which have at least one group which is of a size equal to the group threshold value or which is greater than the threshold value, and to erase those images of the temporal sequence of images which did not influence the generation of such an identified (dilated) binary image.

[0067] Images of the sequence of images which influenced the generation of (dilated) binary images can be identified for example on the basis of a unique identifier of the (dilated) binary images if the unique identifier permits conclusions to be drawn about the original images (e.g. because the unique identifier of the binary images comprises the unique identifier of the original images).

[0068] Those images of the sequence of images which influenced the generation of (dilated) binary images can also be identified for example on the basis of the points in time assigned to the (dilated) binary images.

[0069] The invention can be implemented with the aid of a device. This device according to the invention comprises an input unit, a control unit, a computing unit, an output unit and/or a data storage unit.

[0070] Preferably, the device according to the invention is a computer; it is also conceivable for the device according to the invention to comprise a plurality of computers.

[0071] A “computer” is a device for electronic data processing which processes data by means of programmable computation rules. Such a device usually comprises a motherboard, that unit which comprises a processor for carrying out logic operations, and also peripherals.

[0072] In computer technology, “peripherals” denotes all devices which are connected to the computer and serve for control of the computer and/or as input and output units. Examples are monitor (screen), printer, scanner, mouse, keyboard, drives, camera, microphone, loudspeakers, etc. Internal connections and expansion cards are also deemed to be peripherals in computer technology.

[0073] Present-day computers are often subdivided into desktop PCs, portable PCs, laptops, notebooks, netbooks and tablet PCs and so-called handhelds (e.g. smartphone). The invention can be implemented with all these computers.

[0074] The control unit and the computing unit of the device according to the invention can be for example one or more processors in conjunction with one or more main memories. The input unit can be a (wireless and/or wired) connection to a network or a serial connection (e.g. USB) or the like, via which video material can be transmitted from a camera to the device. The video material is usually stored on a data storage unit (e.g. a hard disk). The input unit can also be used by a user of the device according to the invention in order to effect inputs via a keyboard, a mouse, a microphone, a touchscreen or the like (e.g. the input of one or more threshold values, the number N or the like).

[0075] The result of the processing of the video material according to the invention (the reduced sequence of images) can be stored on the data storage unit (e.g. the hard disk) or output on a monitor.

BRIEF DESCRIPTION OF THE DRAWINGS

[0076] The invention is explained in greater detail below using figures, without any intention for the invention to be restricted to the features and feature combinations shown in the figures.

[0077] FIG. 1 shows a sequence of images schematically and by way of example. The sequence (10) comprises 10 images. The images have a unique identifier: they are numbered consecutively from 1 to 10. Each image of the sequence (10) of images is assigned a point in time. The image having the unique identifier 1 is assigned the point in time 0. The image having the unique identifier 2 is assigned the point in time 20 ms (milliseconds). The image having the unique identifier 3 is assigned the point in time 40 ms. The image having the unique identifier 4 is assigned the point in time 60 ms, and so on. The temporal separation between two images is thus 20 ms. The frame frequency is accordingly 50 frames per second in the present example.

[0078] FIG. 2 shows an image in the form of greyscale raster graphics schematically and by way of example. The raster graphics consist of 10×10=100 pixels. Each pixel can be uniquely addressed on the basis of an x-coordinate and a y-coordinate. Each pixel is assigned a tonal value. The tonal value can specify a greyscale level, for example. In this regard, it is conceivable for each pixel in FIG. 2 to be assigned one of 256 greyscale levels, wherein the tonal value 0 denotes the color tone (the greyscale level) “black” and the tonal value 255 denotes the color tone (the greyscale level) “white”, and the remaining tonal values specify greyscale levels between “black” and “white”. In this regard, in the present example, the pixel having the x,y-coordinates 4,3 has a tonal value of 17, for example, and the pixel having the x,y-coordinates 6,7 has a tonal value of 198, for example.

[0079] FIG. 3 shows by way of example and schematically how a sequence (20) of 9 difference images having the unique identifiers 1-2, 2-3, 3-4, 4-5, 5-6, 6-7, 7-8-, 8-9 and 9-10 is generated from a sequence (10) of 10 images having the unique identifiers 1, 2, 3, 4, 5, 6, 7, 8, 9 and 10.

[0080] The difference images are generated from pairs of temporally directly successive images: a first difference image 1-2 is generated from the images having the identifiers 1 and 2, a second difference image 2-3 is generated from the images having the identifiers 2 and 3, and so on. The unique identifier of a difference image thus yields conclusions drawn from the images from which it was generated.

[0081] Each difference image is assigned a point in time. The difference image having the identifier 1-2 is assigned the point in time 20 ms, the difference image having the identifier 2-3 is assigned the point in time 40 ms, and so on. The frame frequency of the sequence (20) of difference images corresponds to the frame frequency of the sequence (10) of difference images; it is 50 frames per second.

[0082] FIG. 4 shows by way of example and schematically how a difference image having the identifier 1-2 is generated from an image having the identifier 1 and an image having the identifier 2, temporally directly succeeding the image having the identifier 1.

[0083] The image having the identifier 1, the image having the identifier 2 and the difference image having the identifier 1-2 each consist of 5×5=25 pixels. Each pixel is uniquely addressable by coordinates. Each pixel is assigned a tonal value. During the generation of the difference image, the absolute value of the difference in the tonal value of a pixel and of the corresponding pixel of the directly succeeding image is calculated and set as the tonal value of the corresponding pixel of the difference image. Pixels correspond to one another if they have the same coordinates. The tonal value T.sup.1-2(4,3) of the pixel having the coordinates 4,3 of the difference image 1-2 results for example from the tonal value T.sup.1(4,3) of the pixel having the coordinates 4,3 of the image having the identifier 1 and the tonal value T.sup.2(4,3) of the pixel having the coordinates 4,3 of the image having the identifier 2:

T.sup.1-2(4,3)=|T.sup.1(4,3)−T.sup.2(4,3)|=|17−255|=|−238|=238

[0084] Generally it holds true that: T.sup.n-(n+1(x,y)=|T.sup.n(x,y)−T.sup.(n+1)(x,y) wherein n and (n+1) represent the unique identifiers of two directly successive images, n-(n+1) represents the unique identifier of the difference image, and x,y represent the coordinates of the pixels corresponding to one another.

[0085] FIG. 5 shows by way of example and schematically how a sequence (30) of average-value difference images having the unique identifiers 1-2-3-4, 2-3-4-5, 3-4-5-6, 4-5-6-7, 5-6-7-8, 6-7-8-9 and 7-8-9-10 is generated from a sequence (20) of difference images having the unique identifiers 1-2, 2-3, 3-4, 4-5, 5-6, 6-7, 7-8, 8-9 and 9-10.

[0086] Each average-value difference image is assigned a point in time. The average-value difference image having the identifier 1-2-3-4 is assigned the point in time 40 ms, the difference image having the identifier 2-3-4-5 is assigned the point in time 60 ms, and so on. The frame frequency of the sequence (30) of average-value difference images corresponds to the frame frequency of the sequence (20) of difference images; it is 50 frames per second.

[0087] A time window T represented by a pair of brackets is depicted in FIG. 5. The time window T has a length of 60 ms. It can therefore accommodate three directly successive difference images.

[0088] At the beginning the time window T is set to the start of the sequence (20) of difference images. A first average-value difference image is generated from the difference images encompassed by the time window: these are the difference images having the identifiers 1-2, 2-3 and 3-4. In a next step, the time window T is shifted toward the right by one difference image. Now (see the dashed pair of brackets) the time window encompasses the difference images having the identifiers 2-3, 3-4 and 4-5. A second average-value difference image is generated from these difference images. Afterward, the time window is again shifted toward the right by one difference image and a third average-value difference image is generated from the difference images then encompassed by the time window. The process is continued (shifting the time window image by image, generating an average-value difference image from the difference images encompassed by the time window) until all difference images have influenced a generation of an average-value difference image at least once.

[0089] FIG. 6 shows by way of example and schematically how an average-value difference image having the identifier 1-2-3-4 is generated from a difference image having the identifier 1-2, a difference image having the identifier 2-3, directly succeeding the difference image having the identifier 1-2, and a difference image having the identifier 3-4, directly succeeding the difference image having the identifier 2-3.

[0090] The difference image having the identifier 1-2, the difference image having the identifier 2-3, the difference image having the identifier 3-4 and the average-value difference image having the identifier 1-2-3-4 each consist of 5×5=25 pixels. Each pixel is uniquely addressable by coordinates. Each pixel is assigned a tonal value. During the generation, for each pixel of the average-value difference image the arithmetic mean—rounded to an integer—of the tonal values of the corresponding pixels of the difference images is calculated and set as the tonal value of the pixel of the average-value difference image. Pixels correspond to one another if they have the same coordinates. Rounding can involve always rounding up to the nearest integer or always rounding down to the nearest integer or defining a threshold value at which rounding up or down commences.

[0091] In the present example, the tonal value T.sup.1-2-3-4(3,5) of the pixel having the coordinates 3,5 of the average-value difference image 1-2-3-4 results from the tonal value T.sup.1-2(3,5) of the pixel having the coordinates 3,5 of the difference image having the identifier 1-2 and the tonal value T.sup.2-3(3,5) of the pixel having the coordinates 3,5 of the difference image having the identifier 2-3 and the tonal value T.sup.3-4(3,5) of the pixel having the coordinates 3,5 of the difference image having the identifier 3-4:

T.sup.1-2-3-4(3,5)=INT[T.sup.1-2(3,5)+T.sup.2-3(3,5)+T.sup.3-4(3,5))/3]=INT[(192+190+190)/3]=INT[190.6]=191.

[0092] The function INT[ ] rounds a non-integer up or down to the nearest integer (depending on which value is nearer); rounding up is effected in the middle (0.5).

[0093] Generally it holds true that: T.sup.n-(n+1)−(n+2)−n+3) (x,y)=INT[T.sup.n(x,y)+T.sup.(n+1)(x,y)+T.sup.(n+2)(x,y))/3]

[0094] In the example shown in FIGS. 5 and 6, the average-value difference images were generated from three directly successive difference images in each case. It is conceivable, of course, to generate the average-value difference images from a different number of directly successive difference images. The number N of difference images over which averaging is effected is usually greater than 3.

[0095] FIG. 7 shows by way of example and schematically how a sequence (30′) of contrast-reduced average-value difference images having the unique identifiers K-1-2-3-4, K-2-3-4-5, K-3-4-5-6, K-4-5-6-7, K-5-6-7-8, K-6-7-8-9 and K-7-8-9-10 is generated from a sequence (30) of average-value difference images having the unique identifiers 1-2-3-4, 2-3-4-5, 3-4-5-6, 4-5-6-7, 5-6-7-8, 6-7-8-9 and 7-8-9-10.

[0096] Each contrast-reduced average-value difference image is assigned a point in time. The contrast-reduced average-value difference image having the identifier K-1-2-3-4 is assigned the point in time 40 ms, the contrast-reduced average-value difference image having the identifier K-2-3-4-5 is assigned the point in time 60 ms, and so on. The frame frequency of the sequence (30′) of contrast-reduced average-value difference images corresponds to the frame frequency of the sequence (30) of average-value difference images; it is 50 frames per second.

[0097] FIG. 8 shows by way of example and schematically how a sequence (40) of binary images having the unique identifiers B-1-2-3-4, B-2-3-4-5, B-3-4-5-6, B-4-5-6-7, B-5-6-7-8, B-6-7-8-9 and B-7-8-9-10 is generated from a sequence (30) of average-value difference images having the unique identifiers 1-2-3-4, 2-3-4-5, 3-4-5-6, 4-5-6-7, 5-6-7-8, 6-7-8-9 and 7-8-9-10.

[0098] Each binary image is assigned a point in time. The binary image having the identifier B-1-2-3-4 is assigned the point in time 40 ms, the binary image having the identifier B-2-3-4-5 is assigned the point in time 60 ms, and so on. The frame frequency of the sequence (40) of binary images corresponds to the frame frequency of the sequence (30) of average-value difference images; it is 50 frames per second.

[0099] It is likewise conceivable for a sequence of binary images having the unique identifiers B-1-2-3-4, B-2-3-4-5, B-3-4-5-6, B-4-5-6-7, B-5-6-7-8, B-6-7-8-9 and B-7-8-9-10 to be generated from a sequence of contrast-reduced average-value difference images having the unique identifiers K-1-2-3-4, K-2-3-4-5, K-3-4-5-6, K-4-5-6-7, K-5-6-7-8, K-6-7-8-9 and K-7-8-9-10.

[0100] FIG. 9 shows by way of example and schematically how a binary image having the unique identifier B-1-2-3-4 is generated from an average-value difference image having the unique identifier 1-2-3-4.

[0101] The average-value difference image having the identifier 1-2-3-4 and the binary image having the identifier B-1-2-3-4 each consist of 5×5=25 pixels. Each pixel is uniquely addressable by coordinates. Each pixel is assigned a tonal value. During the generation of the binary image, the tonal value of each pixel of the average-value difference image is compared with a tonal-value threshold value TS. If the tonal value of a pixel is less than the tonal-value threshold value TS, then the tonal value of the pixel is set to a first tonal value; if the tonal value of the pixel is greater than the tonal-value threshold value TS or the tonal value of the pixel corresponds to the tonal-value threshold value TS, then the tonal value of the pixel is set to a second tonal value. In the present example, the tonal-value threshold value TS=60, the first tonal-value threshold value is 0 and the second tonal-value threshold value is 255.

[0102] The tonal value of the pixel having the coordinates 4,1 of the average-value difference image is 63. This value is greater than the tonal-value threshold value TS=60. Therefore, the tonal value of the pixel having the coordinates 4,1 of the binary image is set to 255.

[0103] FIG. 10 shows by way of example and schematically how a dilated binary image is generated from a binary image.

[0104] The starting point is the binary image B in FIG. 10(a). The binary image B consists of 10×10=100 pixels. Each pixel is uniquely addressable by its coordinates. Each pixel is assigned a tonal value. There are only two tonal values, the tonal values 0 (“black”) and 1 (“white”).

[0105] A dilatation operator DO is applied to the binary image B. Said dilatation operator expands all pixels having the tonal value “white” to a matrix having 3×3 pixels having the tonal value “white”. As a result, the white pixel having the coordinates 2,9 becomes a matrix having 9 white pixels; the pixels (having the coordinates 1,10; 2,10; 3,10; 1,9; 3,9; 1,8; 2,8; 3,8) directly adjacent to the pixel having the coordinates 2,9 likewise become white irrespective of what tonal value they have before (see FIG. 10(b)).

[0106] It is conceivable for the white pixels to be expanded to a different structuring element than a (3×3) matrix. It is conceivable for such a dilatation operator to be applied multiply.

[0107] FIG. 11 shows by way of example and schematically how it is possible to identify groups of contiguous pixels in a binary image and to ascertain their sizes.

[0108] The starting point is the binary image B in FIG. 11(a). The binary image B consists of 10×10=100 pixels. Each pixel is uniquely addressable by its coordinates. Each pixel is assigned a tonal value. There are only two tonal values, the tonal values 0 (“black”) and 1 (“white”).

[0109] In a first step it is necessary to identify groups of contiguous pixels having the tonal value “white”. Two groups of contiguous pixels having the tonal value “white” are to be identified in the binary image B. Two or more pixels are contiguous if each pixel has at least one directly adjacent pixel having the same tonal value (here “white”). The two identified groups are each provided with a white rectangular bounding border in FIG. 11(b).

[0110] In a further step it is necessary to ascertain the sizes of the groups. One possible method is illustrated by way of example in FIG. 10. In this case, each group of contiguous pixels is provided with a bounding border (see FIG. 11(b)). The bounding border satisfies all the following criteria:

the bounding border is rectangular,
its edges run parallel to the edges of the binary image,
all pixels which have the tonal value “white” and which belong to a group of contiguous pixels lie within the bounding border,
the bounding border comprises as few pixels as possible which do not belong to the group of contiguous pixels having the tonal value “white”.

[0111] The size of a group can be specified as the number of all pixels encompassed by a bounding border. In the present case, the one bounding border encompasses four pixels and the other bounding border encompasses 20 pixels.

[0112] In a further step, the sizes of the areas are compared with a group threshold value FS. In the present example, the group threshold value FS=15 pixels. Therefore, the group having 4 pixels is smaller than the group threshold value FS and the group having 20 pixels is greater than the group threshold value FS.

[0113] Those groups which are of a size at least exactly equal to the group threshold value FS are of interest, then. That is the group having 20 pixels in FIG. 11(c).

[0114] FIG. 12 shows by way of example and schematically the identification of those binary images in a sequence of binary images in which not a single group of contiguous pixels having the second tonal value is present which is greater than the group threshold value.

[0115] Each binary image of the sequence (40) of binary images has a unique identifier: B-1-2-3-4, B-2-3-4-5, B-3-4-5-6, B-4-5-6-7, B-5-6-7-8, B-6-7-8-9 and B-7-8-9-10.

[0116] Each binary image is assigned a point in time. The binary image having the identifier B-1-2-3-4 is assigned the point in time 40 ms, the binary image having the identifier B-2-3-4-5 is assigned the point in time 60 ms, and so on.

[0117] For each binary image a check was made in a preceding step to establish whether it contains at least one group of contiguous pixels having the second tonal value which is of a size equal to a defined group threshold value or which is greater than the defined group threshold value. In the present example, the binary images having the identifiers B-1-2-3-4, B-2-3-4-5 and B-3-4-5-6 have at least one group of contiguous pixels having the second tonal value which is of a size equal to the defined group threshold value or which is greater than the defined group threshold value. In the present example, the binary images having the unique identifiers B-4-5-6-7, B-5-6-7-8, B-6-7-8-9 and B-7-8-9-10 do not have a single area of contiguous pixels having a second tonal value which is of a size equal to the defined group threshold value or which is greater than the defined group threshold value.

[0118] It is only if a binary image has at least one group of contiguous pixels having the second tonal value which is of a size at least equal to the defined group threshold value that the binary image shows a movement which is above a motion threshold and is thus of interest for a more extensive analysis. Binary images which indicate no movement above a motion threshold are of no interest for further evaluation. Those images of the sequence of images which influenced the generation of the binary images which indicate no movement above a motion threshold are therefore erased in a further step.

[0119] In the present example, the images having the unique identifiers 1, 2, 3, 4, 5 and 6 influenced the generation of the binary images having the unique identifiers B-1-2-3-4, B-2-3-4-5 and B-3-4-5-6. These images thus likewise show a movement above a motion threshold and are therefore of interest for a more extensive analysis. The images having the unique identifiers 7, 8, 9 and 10 did not influence the generation of the binary images having the unique identifiers B-1-2-3-4, B-2-3-4-5 and B-3-4-5-6. Instead, they influenced the generation of the binary images having the unique identifiers B-4-5-6-7, B-5-6-7-8, B-6-7-8-9 and B-7-8-9-10. The binary images having the unique identifiers B-4-5-6-7, B-5-6-7-8, B-6-7-8-9 and B-7-8-9-10 show no movement above a motion threshold, however. Therefore, the images having the identifiers 7, 8, 9 and 10 also show no movement above ae motion threshold and are of no interest for a more extensive analysis. The images having the identifiers 7, 8, 9 and 10 are therefore erased.

[0120] This gives rise to a reduced sequence (50) of images, all of which show a movement above a motion threshold.

[0121] FIG. 13 shows one preferred embodiment of the method according to the invention in the form of a flow diagram. The method (100) comprises the steps:

Receiving a temporal sequence of images, wherein each image has a multiplicity of pixels, wherein each pixel is characterized by a tonal value,
Generating a sequence of difference images from the sequence of images, by generating a respective difference image for each pair of adjacent images of the temporal sequence of images, wherein each difference image is characterized by a multiplicity of pixels each having a tonal value, wherein the tonal value of each pixel of each difference image represents an absolute value of the difference between the tonal values of the corresponding pixels of adjacent images,
Generating a sequence of average-value difference images from the sequence of difference images, by a procedure in which a time window is defined which can accommodate a defined number of adjacent difference images, the time window is shifted image by image from the beginning of the sequence of difference images until the end of the sequence of difference images, and, during each shifting image by image, a respective average-value difference image is generated on the basis of the difference images encompassed by the time window, wherein the tonal value of each pixel of each average-value difference image represents an average value of the tonal values of the corresponding pixels of the difference images encompassed by the time window,
Generating a sequence of binary images from the sequence of average-value difference images, by a procedure in which the tonal values of all pixels of each average-value difference image which lie below a defined tonal-value threshold value are set to a first tonal value, and the tonal values of all pixels of each average-value difference image which lie above the defined tonal-value threshold value or correspond to the defined tonal-value threshold value are set to a second tonal value,
Identifying groups of contiguous pixels having a second tonal value in each binary image of the sequence of binary images,
Determining the sizes of the groups for each binary image and comparing the respective size with a group threshold value,
Identifying those binary images which have at least one group which is of a size equal to the group threshold value or which is greater than the group threshold value,
Erasing all images of the temporal sequence of images which have not influenced the generation of the binary image identified in step g).

[0122] FIG. 14 shows a further preferred embodiment of the method according to the invention in the form of a flow diagram. The method (200) comprises the steps:

[0123] Receiving a temporal sequence of images, wherein each image has a multiplicity of pixels, wherein each pixel is characterized by a tonal value,

[0124] Generating a sequence of difference images from the sequence of images, by generating a respective difference image for each pair of adjacent images of the temporal sequence of images, wherein each difference image is characterized by a multiplicity of pixels each having a tonal value, wherein the tonal value of each pixel of each difference image represents an absolute value of the difference between the tonal values of the corresponding pixels of adjacent images,

[0125] Generating a sequence of average-value difference images from the sequence of difference images, by a procedure in which a time window is defined which can accommodate a defined number of adjacent difference images, the time window is shifted image by image from the beginning of the sequence of difference images until the end of the sequence of difference images, and, during each shifting image by image, a respective average-value difference image is generated on the basis of the difference images encompassed by the time window, wherein the tonal value of each pixel of each average-value difference image represents an average value of the tonal values of the corresponding pixels of the difference images encompassed by the time window,

[0126] Generating a sequence of contrast-reduced average-value difference images from the sequence of average-value difference images by applying a Gaussian blur to all average-value difference images of the sequence of average-value difference images,

[0127] Generating a sequence of binary images from the sequence of contrast-reduced average-value difference images, by a procedure in which the tonal values of all pixels of each contrast-reduced average-value difference image which lie below a defined tonal-value threshold value are set to a first tonal value, and the tonal values of all pixels of each contrast-reduced average-value difference image which lie above the defined tonal-value threshold value or correspond to the defined tonal-value threshold value are set to a second tonal value,

[0128] Generating a sequence of dilated binary images from the sequence of binary images, by a procedure in which those pixels of each binary image which have the second tonal value are expanded singly or multiply to a shape of a defined structuring element,

[0129] Identifying groups of contiguous pixels having a second tonal value in each dilated binary image of the sequence of dilated binary images,

[0130] Determining the sizes of the groups for each dilated binary image and comparing the respective size with a group threshold value,

[0131] Identifying those dilated binary images which have at least one group which is of a size equal to the group threshold value or which is greater than the group threshold value,

[0132] Erasing all images of the temporal sequence of images which have not influenced the generation of a dilated binary image identified in step i).

[0133] FIG. 15 shows by way of example and schematically one preferred embodiment of the device according to the invention.

[0134] The device (300) comprises an input unit (301), a control unit (302), a computing unit (303), an output unit (304) and a data storage unit (305).

[0135] The control unit (302) is configured to cause the input unit (301) to receive a sequence of images.

[0136] The control unit (302) is configured to cause the computing unit

[0137] to generate a sequence of difference images from the sequence of images,

[0138] to generate a sequence of average-value difference images from the sequence of difference images,

[0139] to generate a sequence of contrast-reduced average-value difference images from the sequence of average-value difference images,

[0140] to generate a sequence of binary images from the sequence of average-value difference images or the sequence of contrast-reduced average-value difference images,

[0141] to generate a sequence of dilated binary images from the sequence of binary images,

[0142] to identify a group of contiguous pixels in the binary images of the sequence of binary images or in the dilated binary images of the sequence of dilated binary images,

[0143] to determine the size of each identified group in the binary images of the sequence of binary images or in the dilated binary images of the sequence of dilated binary images,

[0144] to compare the determined size of each identified group with a threshold value and to identify the binary images or dilated binary images which have at least one group which is of a size equal to the threshold value or which is greater than the threshold value,

[0145] to erase all images of the temporal sequence of images which do not have a single group of a size at least equal to the threshold value, wherein a reduced sequence of images arises.

[0146] The control unit (302) is configured to store the reduced sequence of images in the data storage unit (305) and/or to cause the output unit (304) to output the reduced sequence of images.