Systems, methods, tangible non-transitory computer-readable media, and devices for autonomous vehicle operation are provided. For example, a computing system can receive object data that includes portions of sensor data. The computing system can determine, in a first stage of a multiple stage classification using hardware components, one or more first stage characteristics of the portions of sensor data based on a first machine-learned model. In a second stage of the multiple stage classification, the computing system can determine second stage characteristics of the portions of sensor data based on a second machine-learned model. The computing system can generate an object output based on the first stage characteristics and the second stage characteristics. The object output can include indications associated with detection of objects in the portions of sensor data.

Systems, methods, tangible non-transitory computer-readable media, and devices for autonomous vehicle operation are provided. For example, a computing system can receive object data that includes portions of sensor data. The computing system can determine, in a first stage of a multiple stage classification using hardware components, one or more first stage characteristics of the portions of sensor data based on a first machine-learned model. In a second stage of the multiple stage classification, the computing system can determine second stage characteristics of the portions of sensor data based on a second machine-learned model. The computing system can generate an object output based on the first stage characteristics and the second stage characteristics. The object output can include indications associated with detection of objects in the portions of sensor data.

An image processing device of a person detection system mounted on a moving body is configured to: detect, in image data obtained from a camera, an area in which an obstacle appears; determine whether the area meets an upper body detection process condition that the obstacle in the area is distanced from a road surface within a predetermined range from the camera; perform an upper body detection process in which the area of the image data is compared with upper body comparison data to determine whether the obstacle in the area is a person, for the area that meets the upper body detection process condition; and perform a whole-body detection process in which the area of the image data is compared with whole-body comparison data to determine whether the obstacle in the area is a person, for the area that does not meet the upper body detection process condition.

An image processing device of a person detection system mounted on a moving body is configured to: detect, in image data obtained from a camera, an area of the image data in which an obstacle appears; perform a center-of-gravity area-width adjustment process in which a position of center of gravity of the obstacle in the area of the image data is estimated and a width of the area of the image data is adjusted based on the position of center of gravity; and determine whether the obstacle in the area is a person by comparing a post-adjustment area obtained after the center-of-gravity area-width adjustment process is performed with dictionary data.

Techniques are provided for recognition of activity in a sequence of video image frames that include depth information. A methodology embodying the techniques includes segmenting each of the received image frames into a multiple windows and generating spatio-temporal image cells from groupings of windows from a selected sub-sequence of the frames. The method also includes calculating a four dimensional (4D) optical flow vector for each of the pixels of each of the image cells and calculating a three dimensional (3D) angular representation from each of the optical flow vectors. The method further includes generating a classification feature for each of the image cells based on a histogram of the 3D angular representations of the pixels in that image cell. The classification features are then provided to a recognition classifier configured to recognize the type of activity depicted in the video sequence, based on the generated classification features.

Techniques are provided for recognition of activity in a sequence of video image frames that include depth information. A methodology embodying the techniques includes segmenting each of the received image frames into a multiple windows and generating spatio-temporal image cells from groupings of windows from a selected sub-sequence of the frames. The method also includes calculating a four dimensional (4D) optical flow vector for each of the pixels of each of the image cells and calculating a three dimensional (3D) angular representation from each of the optical flow vectors. The method further includes generating a classification feature for each of the image cells based on a histogram of the 3D angular representations of the pixels in that image cell. The classification features are then provided to a recognition classifier configured to recognize the type of activity depicted in the video sequence, based on the generated classification features.

A histogram is calculated based on a road surface image of a portion around a vehicle, a running-allowed region in which the vehicle can run is detected based on the histogram, an obstacle region is extracted based on the running-allowed region, and a position of an obstacle in the obstacle region is detected, to further enhance the accuracy of detecting an obstacle around the vehicle as compared with conventional art.

This invention provides a system and method for finding patterns in images that incorporates neural net classifiers. A pattern finding tool is coupled with a classifier that can be run before or after the tool to have labeled pattern results with sub-pixel accuracy. In the case of a pattern finding tool that can detect multiple templates, its performance is improved when a neural net classifier informs the pattern finding tool to work only on a subset of the originally trained templates. Similarly, in the case of a pattern finding tool that initially detects a pattern, a neural network classifier can then determine whether it has found the correct pattern. The neural network can also reconstruct/clean-up an imaged shape, and/or to eliminate pixels less relevant to the shape of interest, therefore reducing the search time, as well significantly increasing the chance of lock on the correct shapes.

This invention provides a system and method for finding patterns in images that incorporates neural net classifiers. A pattern finding tool is coupled with a classifier that can be run before or after the tool to have labeled pattern results with sub-pixel accuracy. In the case of a pattern finding tool that can detect multiple templates, its performance is improved when a neural net classifier informs the pattern finding tool to work only on a subset of the originally trained templates. Similarly, in the case of a pattern finding tool that initially detects a pattern, a neural network classifier can then determine whether it has found the correct pattern. The neural network can also reconstruct/clean-up an imaged shape, and/or to eliminate pixels less relevant to the shape of interest, therefore reducing the search time, as well significantly increasing the chance of lock on the correct shapes.

An image segmentation method according to an embodiment of the present invention is performed in a computing device having one or more processors and memory for storing one or more programs executed by means of the one or more processors, and includes the steps of: (a) receiving the input of an image; (b) generating a first-generation image segment set by dividing the input image in an overlapped manner; and (c) generating a second or higher-generation image segment set from the first-generation image segment set, wherein a subsequent-generation image segment set is generated by dividing in an overlapped manner at least one of a plurality of image segments included in the previous-generation image segment set.