A long-baseline and long depth-range stereo vision system is provided that is suitable for use in non-rigid assemblies where relative motion between two or more cameras of the system does not degrade estimates of a depth map. The stereo vision system may include a processor that tracks camera parameters as a function of time to rectify images from the cameras even during fast and slow perturbations to camera positions. Factory calibration of the system is not needed, and manual calibration during regular operation is not needed, thus simplifying manufacturing of the system.

A surface estimation method includes a region-setting step and an estimation step. In the region-setting step, a reference region that is one of a three-dimensional region and a two-dimensional region is set. The three-dimensional region includes three or more points and is set in a three-dimensional space. The three-dimensional space includes three-dimensional coordinates of three or more points on a subject calculated on the basis of a two-dimensional image of the subject. The three-dimensional coordinates of the three or more points are included in three-dimensional image data. The two-dimensional region includes three or more points and is set in the two-dimensional image. In the estimation step, a reference surface that approximates a surface of the subject is estimated on the basis of three or more points of the three-dimensional image data corresponding to the three or more points included in the reference region.

A surface estimation method includes a region-setting step and an estimation step. In the region-setting step, a reference region that is one of a three-dimensional region and a two-dimensional region is set. The three-dimensional region includes three or more points and is set in a three-dimensional space. The three-dimensional space includes three-dimensional coordinates of three or more points on a subject calculated on the basis of a two-dimensional image of the subject. The three-dimensional coordinates of the three or more points are included in three-dimensional image data. The two-dimensional region includes three or more points and is set in the two-dimensional image. In the estimation step, a reference surface that approximates a surface of the subject is estimated on the basis of three or more points of the three-dimensional image data corresponding to the three or more points included in the reference region.

A three-dimensional model generation method executed by an information processing device includes: obtaining images generated by shooting a subject from respective viewpoints; searching for a similar point that is similar to a first point in a first image among the images, from second points in a search area in a second image different from the first image, the search area being provided based on the first point; calculating an accuracy of a search result of the searching, using degrees of similarity between the first point and the respective second points; and generating a three-dimensional model using the search result and the accuracy.

A novel disparity computation technique is presented which comprises multiple orthogonal disparity maps, generated from approximately orthogonal decomposition feature spaces, collaboratively generating a composite disparity map. Using an approximately orthogonal feature set extracted from such feature spaces produces an approximately orthogonal set of disparity maps that can be composited together to produce a final disparity map. Various methods for dimensioning scenes and are presented. One approach extracts the top and bottom vertices of a cuboid, along with the set of lines, whose intersections define such points. It then defines a unique box from these two intersections as well as the associated lines. Orthographic projection is then attempted, to recenter the box perspective. This is followed by the extraction of the three-dimensional information that is associated with the box, and finally, the dimensions of the box are computed. The same concepts can apply to hallways, rooms, and any other object.

A novel disparity computation technique is presented which comprises multiple orthogonal disparity maps, generated from approximately orthogonal decomposition feature spaces, collaboratively generating a composite disparity map. Using an approximately orthogonal feature set extracted from such feature spaces produces an approximately orthogonal set of disparity maps that can be composited together to produce a final disparity map. Various methods for dimensioning scenes and are presented. One approach extracts the top and bottom vertices of a cuboid, along with the set of lines, whose intersections define such points. It then defines a unique box from these two intersections as well as the associated lines. Orthographic projection is then attempted, to recenter the box perspective. This is followed by the extraction of the three-dimensional information that is associated with the box, and finally, the dimensions of the box are computed. The same concepts can apply to hallways, rooms, and any other object.

Techniques are provided for recognition of activity in a sequence of video image frames that include depth information. A methodology embodying the techniques includes segmenting each of the received image frames into a multiple windows and generating spatio-temporal image cells from groupings of windows from a selected sub-sequence of the frames. The method also includes calculating a four dimensional (4D) optical flow vector for each of the pixels of each of the image cells and calculating a three dimensional (3D) angular representation from each of the optical flow vectors. The method further includes generating a classification feature for each of the image cells based on a histogram of the 3D angular representations of the pixels in that image cell. The classification features are then provided to a recognition classifier configured to recognize the type of activity depicted in the video sequence, based on the generated classification features.

A method and an apparatus optimizes scan data obtained by sensors on vehicle, and corrects trajectory for a vehicle/robot based on the optimized scan data. The method for optimizing the scan data obtained by scanning environment elements, includes: step of obtaining the scan data, including obtaining at least two frames of scan data respectively corresponding to different timings; step of cluster processing, based on the characteristic of the data points, including classifying the plurality of data points in each frame of the scan data into one or more clusters; step of establishing correspondence, among the at least two frames of scan data, including searching and obtaining at least one set of clusters having correspondence; step of optimizing clusters, among the at least two frames of scan data, including conducting calculation to each set of the at least one set of clusters having correspondence, to obtain optimized clusters respectively corresponding to each set of the at least one set of clusters having correspondence; and step of optimizing the scan data, including accumulating all optimized clusters to obtain an optimized scan date for the at least two frames of scan data.

A method and an apparatus optimizes scan data obtained by sensors on vehicle, and corrects trajectory for a vehicle/robot based on the optimized scan data. The method for optimizing the scan data obtained by scanning environment elements, includes: step of obtaining the scan data, including obtaining at least two frames of scan data respectively corresponding to different timings; step of cluster processing, based on the characteristic of the data points, including classifying the plurality of data points in each frame of the scan data into one or more clusters; step of establishing correspondence, among the at least two frames of scan data, including searching and obtaining at least one set of clusters having correspondence; step of optimizing clusters, among the at least two frames of scan data, including conducting calculation to each set of the at least one set of clusters having correspondence, to obtain optimized clusters respectively corresponding to each set of the at least one set of clusters having correspondence; and step of optimizing the scan data, including accumulating all optimized clusters to obtain an optimized scan date for the at least two frames of scan data.

A sensing and computing system and method for capturing images and data regarding an object and calculating one or more parameters regarding the object using an internal, integrated CPU/GPU. The system comprises an imaging system, including a depth imaging system, color camera, and light source, that capture images of the object and sends data or signals relating to the images to the CPU/GPU, which performs calculations based on those signals/data according to pre-programmed algorithms to determine the parameters. The CPU/GPU and imaging system are contained within a protective housing. The CPU/GPU transmits information regarding the parameters, rather than raw data/signals, to one or more external devices to perform tasks in an industrial environment related to the object imaged.