Method and device for generation of a representation of a digital image

Abstract

A method and device for real-time generation of a multiresolution representation of a digital image for real-time generation are disclosed. A sequence of main representations of the digital image is stored at successive different main resolutions in a main memory. A part of a current main representation is loaded from the main memory into a local memory via a bus. A current main representation is processed by determining a corresponding part of an intermediate representation of the image having an intermediate resolution lying between the resolution of the current main representation and the resolution of the subsequent main representation. The loading and processing steps are repeated for other parts of the current main representation until all parts of the current main representation have been successively loaded and processed.

Claims

1. A method for real-time generation of a multiresolution representation of a digital image, the method comprising: storing a sequence of main representations of the digital image at successive different main resolutions; and processing a current main representation by: a) loading a part of the current main representation into a local memory via a bus; b) scanning the loaded part of the current main representation using a rectangular detection window; c) at each position in the scan, calculating an integral image of a portion of the digital image contained in the detection window; d) determining at least one corresponding part of at least one intermediate representation of the digital image having an intermediate resolution lying between the resolution of the current main representation and the resolution of the subsequent main representation, scanning at least a corresponding part of the intermediate representation by using the rectangular detection window, and calculating, at each position in the scan, an integral image of the portion of the digital image contained in the detection window; e) repeating steps a) to d) with other parts of the current main representation until all parts of the current main representation have been successively loaded and processed; and repeating steps a) to e) for a plurality of subsequent main representations.

2. The method according to claim 1, comprising, for each iteration of steps a) to d), repeating step d) to determine at least a corresponding part of at least one new intermediate representation of the digital image having an intermediate resolution lying between the intermediate resolution of the preceding intermediate representation and the main resolution of the subsequent main representation, causing at least a corresponding part of the at least one new intermediate representation to be scanned using the rectangular detection window, and, at each position in the scan, calculating an integral image of the portion of the digital image contained in said detection window.

3. The method according to claim 1, wherein movement between two positions of the detection window is smaller than a size of the detection window, so that a part of the detection window overlaps the detection window in its preceding position.

4. The method according to claim 1, wherein the detection window is moved in a first direction of the digital image and shifted in a second direction of the digital image, which is orthogonal to the first direction, whenever the detection window has reached an end of the digital image in the first direction, the movement of the detection window in the first direction taking place from one end of the digital image to an opposite end.

5. A method of real-time detection of an object in a digital image using a Viola-Jones method, the method comprising: generating a sequence of main representations of the digital image at different successive main resolutions; performing the method of claim 1 to the main representations of the digital image; and processing the integral images calculated by classifiers executing the Viola-Jones method.

6. A device comprising: a main memory storing a sequence of main representations of a digital image at different main resolutions; a local memory coupled to the main memory via a bus; a processor coupled to the local memory and configured for, for a current main representation, executing instructions to perform the steps of : a) loading a part of the current main representation into the local memory; b) causing the loaded part of the current main representation to be scanned using a rectangular detection window; c) calculating, at each position in the scan, an integral image of a portion of the digital image contained in the detection window; d) determining at least one corresponding part of at least one intermediate representation of the digital image having an intermediate resolution lying between the resolution of the current main representation and the resolution of the subsequent main representation, scanning at least a corresponding part of the at least one intermediate representation using the rectangular detection window, and calculating, at each position in the scan, an integral image of the portion of the digital image contained in the detection window; e) repeating steps a) to d) with various parts of the current main representation until all parts of the current main representation have been successively loaded and processed; and f) repeating steps a) to e) for a subsequent main representation.

7. The device according to claim 6, wherein the processor is further configured to execute instructions to repeat step d) at least once, for each iteration of steps a) to d), in order to determine at least a corresponding part of at least one new intermediate representation of the digital image having an intermediate resolution lying between the intermediate resolution of the preceding intermediate representation and the main resolution of the subsequent main representation, cause at least a corresponding part of the new intermediate representation to be scanned by using the rectangular detection window, and, at each position in the scan, calculate an integral image of the portion of the digital image contained in the detection window.

8. The device according to claim 6, wherein the processor is further configured to execute instructions to cause movement of the detection window between two successive positions so that a part of the detection window overlaps the detection window in its preceding position.

9. The device according to claim 8, wherein the processor is further configured to execute instructions to move the detection window in a first direction of the digital image and to shift it in a second direction of the digital image, which is orthogonal to the first direction, whenever the detection window has reached an end of the digital image in the first direction, the movement of the detection window in the first direction taking place from one end of the digital image to an opposite end.

10. The device according to claim 6, wherein the device utilizes a Viola-Jones method.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) Other advantages and features of the invention will be apparent from an examination of the detailed description of an embodiment of the invention, which is not limiting in any way, and the appended drawings, in which:

(2) FIG. 1 shows schematically a piece of electronic equipment comprising a device for the real-time detection of at least one object in a digital image by the Viola-Jones method, according to an embodiment of the invention; and

(3) FIG. 2 shows schematically some main and intermediate representations of a pyramid of images generated according to an embodiment of the invention.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

(4) FIG. 1 shows schematically a piece of electronic equipment comprising a device for the real-time detection of at least one object in a digital image by the Viola-Jones method, according to an embodiment of the invention.

(5) The electronic equipment APP comprises, for example, an imaging device 1 such as a digital camera or video camera, and a detection device 2 to which the imaging device 1 is coupled.

(6) The detection device 2, intended to detect at least one object in a digital image in real time by the Viola-Jones method, comprises means 3 for the real-time generation of a pyramid of main representations, at successive different main resolutions, of each image received. These means 3 are coupled to the imaging device 1 and are capable of generating a succession of main representations of the image at different main resolutions.

(7) The detection device 2 further comprises a device 4 for the real-time generation of a multiresolution representation of each digital image received from the imaging device 1 by the detection device 2, and classifiers 5 for the execution of the Viola-Jones method. These classifiers are intended to receive the integral images and the main and intermediate representations delivered by the generation device 4, notably at each horizontal or vertical movement of the detection window.

(8) The device 4 comprises a main memory 6, for example a DRAM memory, which is coupled to the output of the means 3 for generating a pyramid of main representations, and is capable of storing, for each image, the main representations of the image at the various main resolutions.

(9) The device 4 further comprises a local memory 7, for example an SRAM memory, coupled via a transfer bus 8 to the main memory 6. The local memory 7 is smaller than the main memory 6. Depending on the embodiment, the local memory 7 is configured to store one main representation of the image at a time, or possibly one part or strip of a main representation of the image at a time, for the purpose of performing a number of subsamplings of the main representation and calculations of integral images for each of the subsamplings.

(10) In order to simplify the generation of a multiresolution representation as far as possible, a full image can effectively be stored in the local memory 7.

(11) However, by storing only one strip of a main representation, updated line by line for example, it is possible to generate the intermediate resolutions between two main resolutions for this image strip. The size of the image strip and the choice of the subsampling factor between two main resolutions make it possible to provide all the possible intermediate resolutions between these two main resolutions.

(12) The device 4 further comprises a processing unit 9, which is configured to generate a rectangular detection window F and to move it over the whole image, and which can calculate an integral image of the detection window at each of its positions.

(13) FIG. 2 shows schematically some main and intermediate representations of a pyramid of images generated according to an embodiment of the invention.

(14) The main representations A, B and C of the image are shown in solid lines. The main representations A, B and C are representations of the processed image generated by the generating means 3. These main representations A, B and C have different main resolutions, for example resolutions of 1, , 1/16, 1/64, and so on. The resolution of corresponds to a resolution of in horizontal terms and in vertical terms. These main representations A, B and C are stored in the main memory 7.

(15) The generation device 4 loads the main representations A, B and C, one by one, into its local memory 7. When one of the main representations A, B or C is loaded into the local memory 7, the main or intermediate representation preceding it in memory is overwritten.

(16) In the example shown in FIG. 2, the main representation A, having a main resolution of 1 for example, is initially loaded into the local memory 7 via the transfer bus 8.

(17) The processing unit 9 then generates a rectangular detection window F, measuring 48 by 48 pixels for example, and calculates the corresponding integral image at each position F.sub.i of the detection window F.

(18) The various functions of the processing unit 9 which will now be described may be executed by software modules within a microprocessor.

(19) The processing unit 9 comprises a positioning software module 10 capable of causing the detection window F to move between two successive positions F.sub.i1 and F.sub.i. The detection window F is moved from a preceding position F.sub.i1 to a subsequent position F.sub.i through a distance smaller than the length of the detection window in the direction of its movement, so that at least a part of the detection window in the subsequent position F.sub.i overlaps the detection window F in its preceding position F.sub.i1.

(20) As shown by the arrows, the detection window F is moved in a horizontal direction of the main representation A, and then in a vertical direction of the main representation A. The detection window F is thus moved downwards from line to line. In the illustrated embodiment, whenever the detection window F is moved in the vertical direction, it is repositioned at the start of a line so as to partially overlap the position of the detection window at the start of the preceding line, corresponding to the position denoted F.sub.i1 in this case.

(21) The processing unit 9 is also configured to execute a subsampling of the content of the local memory 7. More precisely, the processing unit 9 is configured to determine at least one intermediate representation A.sub.1, A.sub.2, B.sub.1, B.sub.2 of the image, in this case two representations as in the example shown in FIG. 2, having an intermediate resolution lying between the resolution of the current main representation, A or B in the illustrated example, and the resolution of the subsequent main representation, B or C respectively, in the illustrated example.

(22) In the illustrated example, the intermediate resolution A.sub.1 is therefore between 1 and .

(23) When the intermediate representation A.sub.1 has been determined, the processing unit 9 is configured to cause the intermediate representation A.sub.1 to be scanned by the rectangular detection window F, and optionally, depending on the nature of the classifier used, and at each position F.sub.i of the scan, to calculate an integral image of the detection window F.

(24) It is possible to determine the representation A.sub.1 in advance, and store it in the local memory 7 before calculating the integral images, or alternatively to determine the representation A.sub.1 and calculate the integral image of each window during the course of the operation.

(25) After the determination of the first intermediate representation A.sub.1 has been completed, the second intermediate representation A.sub.2 is calculated directly on the basis of the main representation A stored in the local memory 7.

(26) The number of strips of intermediate resolution A.sub.1, A.sub.2 . . . Ai that can be generated for this strip of main resolution before a vertical increment of n lines depends on the number of image lines that can be stored in the local memory 7.

(27) In a variant, this intermediate representation A.sub.1 could, for example, be stored fully or partially in the local memory 7 so as to be processed by the processing unit 9 in order to construct at least a part of the second intermediate representation A.sub.2 and determine the integral images for this second intermediate representation A.sub.2.

(28) The intermediate resolution of the intermediate representation A.sub.2 lies between the intermediate resolution of the preceding intermediate representation A.sub.1 and the main resolution of the subsequent main representation B. The calculation of the integral images of the representation A.sub.2 is identical to that performed for the representation A.sub.1.

(29) On completion of this processing, in the example shown in FIG. 1, the generation device 4 loads the second main representation B into the local memory 7 so that it can be processed by the processing unit 9 in order to calculate the integral images of the main representation B and of the two intermediate representations B.sub.1 and B.sub.2 and construct the intermediate representations B.sub.1 and B.sub.2 in themselves.

(30) Then, in the example shown in FIG. 2, the generation device 4 loads a third main representation C into the local memory 7 so that it can be processed by the processing unit 9 in order to calculate the integral images of the main representation C.

(31) The image windows, or integral image windows, calculated for each main representation A, B, C and intermediate representation A.sub.1, A.sub.2, B.sub.1, B.sub.2 are delivered to the classifier 5 of the detection device 2, together with the main representations A, B, C or the intermediate representations A.sub.1, A.sub.2, B.sub.1, B.sub.2 so that the object detection can take place at the different resolutions of the image.

(32) The system and method of detection by the Viola-Jones method, using a method of real-time generation of a multiresolution representation of a digital image based on the generation of intermediate resolutions on the basis of main resolutions, enable the bandwidth used in a transfer bus to be reduced, thereby allowing object detection to be performed on digital images in real time.