Classification of Image Data with Adjustment of the Degree of Granulation

Abstract

A device for classifying image data includes a trainable pre-processing unit configured to retrieve, from a trained context, and based on the image data, at least one specification in terms of how a degree of granulation of the image data is to be reduced, and to reduce the degree of granulation of the image data in accordance with the at least one specification. The device further includes a trainable classifier configured to map the granulation-reduced image data onto an assignment to one or more classes of a specified classification.

Claims

1. A device for classification of image data, comprising: a trainable pre-processing unit configured to retrieve, from a trained relationship, and based on the image data, at least one specification regarding an extent to which a level of detail of the image data is to be reduced, and to reduce the level of detail of the image data in accordance with the at least one specification; and a trainable classifier configured to map the detail-reduced image data onto an assignment to one or more classes of a specified classification.

2. The device according to claim 1, wherein the pre-processing unit and the classifier are configured designed as a common artificial neural network (“ANN”).

3. The device according to claim 1, wherein: the pre-processing unit is connected via a dedicated broadband connection to at least one image source carried by a vehicle, the pre-processing unit is connected to the classifier via a bus system of the vehicle, and the bus system is also used by further on-board systems of the vehicle.

4. The device according to claim 1, wherein the pre-processing unit is configured to: transform the image data into a representation in a working space, and reduce the level of detail of the this representation.

5. The device according to claim 4, wherein the pre-processing unit is configured to: determine the representation in the working space as a linear combination of basic functions of the working space characterized by a set of coefficients, retrieve the at least one specification, based on the image data, from the trained relationship regarding which of the coefficients are to be reduced in terms of magnitude or eliminated, and reduce or eliminate the coefficients of the representation according to the at least one specification.

6. The device according to claim 5, wherein: the pre-processing unit is configured to determine the representation in the working space as a linear combination of wavelets; and the wavelets are characterized by a set of wavelet coefficients.

7. The device according to claim 5, wherein the pre-processing unit is configured to retrieve, as the at least one specification, a numerical fraction of the coefficients which are to be reduced in terms of magnitude or eliminated.

8. The device according to claim 7, wherein the pre-processing unit is configured to determine the numerical fraction of the coefficients which are to be reduced in terms of magnitude or eliminated, using entropy of the image data.

9. The device according to claim 4, wherein the pre-processing unit is configured to: retrieve a dimensionality for a latent space of an autoencoder as a working space using the image data from the trained relationship, and transform the image data into the working space using the autoencoder.

10. A method for training a device, comprising: providing learning image data and associated learning assignments onto which the device is to nominally map the learning image data; defining a specification for reducing a level of detail that is sought on average; and optimizing parameters characterizing a behavior of the trainable pre-processing unit of the device to aims that the device maps the learning image data onto the learning assignments and, at the same time, the reduction in the level of detail which the pre-processing unit performs on the learning image data corresponds, on average, to the specification.

11. The method according to claim 10, wherein parameters characterizing a behavior of the classifier of the device are additionally optimized to an aim that the device maps the learning image data onto the learning assignments.

12. The method according to claim 11, wherein the optimization of the parameters which characterize the behavior of the classifier is additionally also directed to an aim that the level of detail of the image data used by the classifier be as low as possible.

13. The method according to claim 10, wherein a computer program contains machine-readable instructions which, when executed on one or more computers, upgrade the computer or computers to the device and/or cause the method to be carried out.

14. A non-transitory machine-readable data carrier comprising the computer program according to claim 13.

15. A computer comprising the computer program according to claim 13.

Description

EMBODIMENTS

[0045] In the drawings:

[0046] FIG. 1 shows an embodiment of the device 1;

[0047] FIG. 2 shows a further embodiment of the device 1 in a vehicle 6;

[0048] FIG. 3 shows a schematic example of detail reduction taking the example of a pedestrian 4;

[0049] FIG. 4 shows an embodiment of the method 100.

[0050] FIG. 1 shows an embodiment of the device 1 for the classification of image data 2. The device 1 comprises a trainable pre-processing unit 11 and a classifier 12.

[0051] The pre-processing unit 11 receives image data 2 from any source and in a block 111 determines on the basis of these image data 2 at least one specification 3 regarding the extent to which the level of detail of the image data 2 is to be reduced. In block 112, the level of detail of the image data 2 is reduced according to this specification 3, such that detail-reduced image data 4 result. These detail-reduced image data 4 are mapped onto an assignment 5 to one or more classes of a specified classification.

[0052] FIG. 2 shows a further embodiment of the device 1 which is installed in a vehicle 6. The vehicle 6 has four cameras 61a-61d that monitor regions 60a-60d of the surroundings of the vehicle 6. Each of these cameras 61a-61d supplies image data 2 via a dedicated broadband connection 62a-62d to a pre-processing unit 11a-11d that is associated in each case. All pre-processing units 11a-11d are connected to a central bus system 63 of the vehicle 6, which is also used by further on-board systems 64-66 of the vehicle 6. The pre-processing units 11a-11d supply highly compressed detailed-reduced image data 4 to the trainable classifier 12. In this way, the classifier 12 can process the most important details of the image data 2 without the full data stream of the image data 2 having to be transmitted via the bus system 63 of the vehicle 6, the bus system not being designed for this purpose. For the sake of clarity the feedback from the classifier to actuators of the vehicle 6 is not shown in FIG. 2.

[0053] FIG. 3 shows an example of the detail reduction taking the example of a pedestrian 7. FIG. 3a is a schematic view of image data 2 provided by a camera. The pedestrian 7 has a face 71 with pronounced facial features. He is also wearing a t-shirt 72 with an inscription 73 and shoulder pads 74.

[0054] All these details are not relevant to the important recognition that this is a pedestrian. Certain details could even distract a classifier 12. Thus, for example, certain features in the face 71, or a tattered state of the t-shirt 72, could cause the classifier 12 to incorrectly classify the pedestrian 7 as a scarecrow. Likewise, the shoulder pads 74 could cause the classifier 12 to classify the pedestrian 7 as a display dummy. Both cases would be disadvantageous for the pedestrian 7, because a system for at least partially automated driving would assume that only slight material damage occurs in the event of a collision with a scarecrow or a display dummy, and in the case of doubt would give preference to this collision over a collision with another vehicle. The same could happen if the incorrect classification is deliberately brought about by a manipulative “adversarial” pattern 75 in the image data 2, for example by a semi-transparent sticker on the camera lens.

[0055] FIG. 3b shows the detail-reduced image data 4. In this case, it can now be seen only abstractly that this is a pedestrian 7. All further details are omitted, such that additional bandwidth and computing capacity are no longer required for their processing. The “adversarial” pattern 75 is also smoothed to form a homogeneous spot and thus rendered harmless.

[0056] FIG. 4 shows an embodiment of the method 100 for training the device 1. In step 110, learning image data 2a and associated learning assignments 5a are provided. In step 120, a specification 3a for reducing the level of detail in the learning image data 2a, which is sought on average, is determined. In step 130, parameters 11* characterizing the behavior of the trainable pre-processing unit 11, 11a-11d of the device 1 are optimized for the aims of [0057] the device 1 imaging the learning image data 2a onto the learning assignments 5a, and [0058] at the same time the reduction in the level of detail that the pre-processing unit 11, 11a-11d performs on the learning image data 2a corresponds on average to the specification 3a.

[0059] In addition, in step 140 in this embodiment, parameters 12* which characterize the behavior of the classifier 12 of the device 1 are also optimized for the aim of the device 1 mapping the learning image data 2a onto the learning assignments 5a. This training is dovetailed with the training 130 of the parameters 11* of the pre-processing unit 11, 11a-11d.

[0060] According to block 141, optimization 140 of the parameters 12* of the classifier 12 is additionally also directed at the aim of the level of detail of the image data 2 used by the classifier 12 being as low as possible.