METHOD FOR DETECTING AND RE-IDENTIFYING OBJECTS USING A NEURAL NETWORK

Abstract

A method for detecting and re-identifying objects using a neural network. The method includes the steps: extracting features from an image, the features comprising information about at least one object in the image; detecting the at least one object in the image using an anchor-based object detection based on the extracted features, classification data being determined by a classification for detecting the object with the aid of at least one anchor and regression data being determined by a regression; and re-identifying the at least one object by determining embedding data based on the extracted features, the embedding data representing an object description for the at least one feature of the image.

Claims

1. A method for detecting and re-identifying objects using a neural network, the method comprising the following steps: extracting features from an image, the features including information about at least one object in the image; detecting the at least one object in the image using an anchor-based object detection based on the extracted features, classification data being determined by a classification for detecting the object using at least one anchor and regression data being determined by a regression; and re-identifying the at least one object by determining embedding data based on the extracted features, the embedding data representing an object description for the at least one feature of the image.

2. The method as recited in claim 1, further comprising the following step: tracking the at least one object in successive images based on the determined classification data, the determined regression data and the determined embedding data.

3. The method as recited in claim 1, wherein the detecting of the at least one object and the determining of the embedding data occur at the same time based on identical features and using the same neural network.

4. The method as recited in claim 1, wherein the embedding data are determined by an embedding layer, the embedding layer being learned with a loss function.

5. The method as recited in claim 4, wherein the loss function includes a metrics, the metrics including an L2norm or a cosine distance.

6. The method as recited in claim 4, wherein the embedding layer is learned with object re-identification, distances being used between the embedding data of detected objects.

7. The method as recited in claim 4, wherein the embedding layer is learned with temporal detection, the embedding data of detected objects being used as input for a tracking algorithm.

8. A neural network, comprising: a feature extractor configured to extract features from an image, the features including information about at least one object in the image; a classification layer configured to determine classification data for detecting the at least one object in an image using an anchor-based object detection; a regression layer configured to determine regression data for detecting the at least one object in the image using the anchor-based object detection; and an embedding layer configured to re-identify the at least one object by determining embedding data based on the extracted features, the embedding data representing an object description for the at least one feature of the image.

9. A control method for an at least partially autonomous robot, the method comprising the following steps: receiving image data of the at least partially autonomous robot, the image data representing a surroundings of the robot; applying a method for detecting and re-identifying objects to the received image data using a neural network, including: extracting features from the received image data, the features including information about at least one object in the received image data, detecting the at least one object in the received image data using an anchor-based object detection based on the extracted features, classification data being determined by a classification for detecting the object using at least one anchor and regression data being determined by a regression, and re-identifying the at least one object by determining embedding data based on the extracted features, the embedding data representing an object description for the at least one feature of the image data; and controlling the at least partially autonomous robot based on the detected and re-identified at least one object.

10. A non-transitory machine-readable storage medium on which is stored a computer program for detecting and re-identifying objects using a neural network, the computer program, when executed by a computer, causing the computer to perform the following steps: extracting features from an image, the features including information about at least one object in the image; detecting the at least one object in the image using an anchor-based object detection based on the extracted features, classification data being determined by a classification for detecting the object using at least one anchor and regression data being determined by a regression; and re-identifying the at least one object by determining embedding data based on the extracted features, the embedding data representing an object description for the at least one feature of the image.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0062] FIG. 1 shows a neural network for detecting and re-identifying objects in an image.

[0063] FIG. 2 shows a method for detecting and re-identifying objects in an image.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

[0064] FIG. 1 shows a neural network 10 for detecting and re-identifying objects in an image. Neural network 10 comprises a feature extractor 20, which is designed to extract features F from an image, features F comprising information about at least one object in the image.

[0065] The neural network further comprises a classification layer 30, a regressions layer 40, an embedding layer 50 and a tracking algorithm 60. Feature extractor 20 supplies classification layer 30, regression layer 40 and embedding layer 50 with the extracted features F. Classification layer 30 is designed to classify at least one object in the image using an anchor-based object detection and to determine classification data Dk in the process. Regression layer 40 is designed to detect at least one object in the image using an anchor-based object detection and to determine regression data Dr in the process. Embedding layer 50 is designed to determine at least one embedding vector V based on the extracted features F in order to re-identify the at least one object in multiple successive images, the at least one embedding vector representing an object description for the at least one feature F of the image.

[0066] The determined classification data Dk, regression data Dr and the at least one embedding vector V are subsequently transmitted to tracking algorithm 60. Tracking algorithm 60 is designed to track the at least one object in successive images based on the determined classification data Dk, the determined regression data Dr and the determined at least one embedding vector V.

[0067] FIG. 2 shows a method for detecting and re-identifying objects using a neural network, comprising the following steps. In a first step S10, features F are extracted from an image, features F comprising information about at least one object in the image. In a second step S20, the at least one object in the image is detected using an anchor-based object detection based on the extracted features F, classification data Dk being determined by a classification for detecting the object with the aid of at least one anchor and regression data Dr being determined by a regression. In a third step S30, the at least one object is re-identified by determining at least one embedding vector V on the basis of the extracted features F, the at least one embedding vector V representing an object description for the at least one feature F of the image.