A network for category-level 6D pose and size estimation, including a 3D-OCR module for 3D Orientation-Consistent Representation, a GeoReS module for Geometry-constrained Reflection Symmetry, and a MPDE module for Mirror-Paired Dimensional Estimation; wherein the 3D-OCR module and the GeoReS module are incorporated in parallel; the 3D-OCR module receives a canonical template shape including canonical category-specific keypoints; the GeoReS module receives an original input depth observation including pre-processed predicted category labels and potential masks of the target instances; the MPDE module receives the output from the GeoReS module as well as the original input depth observation; and the network outputs the estimation results based on the output of the MPDE module, the output of the 3D-OCR module, as well as the canonical template shape. Also provided are corresponding systems and methods.

A system configured to assist a user in scanning a physical environment in order to generate a three-dimensional scan or model. In some cases, the system may include an interface to assist the user in capturing data usable to determine a scale or depth of the physical environment and to perform a scan in a manner that minimizes gaps.

A system configured to assist a user in scanning a physical environment in order to generate a three-dimensional scan or model. In some cases, the system may include an interface to assist the user in capturing data usable to determine a scale or depth of the physical environment and to perform a scan in a manner that minimizes gaps.

A three-dimensional data encoding method includes: extracting, from first three-dimensional data, second three-dimensional data having an amount of a feature greater than or equal to a threshold; and encoding the second three-dimensional data to generate first encoded three-dimensional data. For example, the three-dimensional data encoding method may further include encoding the first three-dimensional data to generate the second encoded three-dimensional data.

A three-dimensional data encoding method includes: extracting, from first three-dimensional data, second three-dimensional data having an amount of a feature greater than or equal to a threshold; and encoding the second three-dimensional data to generate first encoded three-dimensional data. For example, the three-dimensional data encoding method may further include encoding the first three-dimensional data to generate the second encoded three-dimensional data.

Techniques to facilitate compression of depth data and real-time reconstruction of high-quality light fields. A parameter space of values for a line, pairs of endpoints on different sides of the line, and a palette index for each pixel of a pixel tile of a depth image is sampled. Values for the line, the pairs of endpoints, and the palette index that minimize an error are determined and stored.

A computer system maintains structure data indicating geometrical constraints for each structure category of a plurality of structure categories. The computer system generates a virtual 3D representation of a structure based on a set of images depicting the structure. For each image in the set of images, one or more landmarks are identified. Based on the landmarks, a candidate structure category is selected. The virtual 3D representation is generated based on the geometrical constraints of the candidate structure category and the landmarks identified in the set of images.

A computer system maintains structure data indicating geometrical constraints for each structure category of a plurality of structure categories. The computer system generates a virtual 3D representation of a structure based on a set of images depicting the structure. For each image in the set of images, one or more landmarks are identified. Based on the landmarks, a candidate structure category is selected. The virtual 3D representation is generated based on the geometrical constraints of the candidate structure category and the landmarks identified in the set of images.

A visualization system including multiple light sources, an image sensor configured to detect imaging data from the multiple light sources, and a control circuit is disclosed. At least one of the light sources is configured to emit a pattern of structured light. The control circuit is configured to receive the imaging data from the image sensor, generate a three-dimensional digital representation of the anatomical structure from the pattern of structured light detected by the imaging data, obtain metadata from the imaging data, overlay the metadata on the three-dimensional digital representation, receive updated imaging data from the image sensor, and generate an updated three-dimensional digital representation of the anatomical structure based on the updated imaging data. The visualization system can be communicatively coupled to a situational awareness module configured to determine a surgical scenario based on input signals from multiple surgical devices.

A visualization system including multiple light sources, an image sensor configured to detect imaging data from the multiple light sources, and a control circuit is disclosed. At least one of the light sources is configured to emit a pattern of structured light. The control circuit is configured to receive the imaging data from the image sensor, generate a three-dimensional digital representation of the anatomical structure from the pattern of structured light detected by the imaging data, obtain metadata from the imaging data, overlay the metadata on the three-dimensional digital representation, receive updated imaging data from the image sensor, and generate an updated three-dimensional digital representation of the anatomical structure based on the updated imaging data. The visualization system can be communicatively coupled to a situational awareness module configured to determine a surgical scenario based on input signals from multiple surgical devices.