The invention relates to a method and a system for determining a pose of at least one object in an operating theatre, in a reference coordinate system of a pose detection device of a surgical microscope, involving the determination of the pose of the object by way of a movably arranged microscope-external pose detection device in a first coordinate system, the first coordinate system being a coordinate system that is arranged to be stationary relative to the operating theatre, the determination of the pose of the reference coordinate system by the non-stationary microscope-external pose detection device in the first coordinate system, and the transformation of the pose of the object from the first coordinate system into the reference coordinate system of the pose detection device of the surgical microscope.

The invention relates to a method and a system for determining a pose of at least one object in an operating theatre, in a reference coordinate system of a pose detection device of a surgical microscope, involving the determination of the pose of the object by way of a movably arranged microscope-external pose detection device in a first coordinate system, the first coordinate system being a coordinate system that is arranged to be stationary relative to the operating theatre, the determination of the pose of the reference coordinate system by the non-stationary microscope-external pose detection device in the first coordinate system, and the transformation of the pose of the object from the first coordinate system into the reference coordinate system of the pose detection device of the surgical microscope.

Provided are a depth completion method and apparatus using spatial-temporal information. The depth completion apparatus according to the present invention comprises a processor; and a memory connected to the processor, wherein the memory stores program instructions executable by the processor for performing operations comprising receiving an RGB image and a sparse image through a camera and LiDAR, generating a dense first depth map by processing color information of the RGB image through a first branch based on an encoder-decoder, generating a dense second depth map by up-sampling the sparse image through a second branch based on an encoder-decoder, generating a third depth map by fusing the first depth map and the second depth map, and generating a final depth map including a trajectory of a moving object included in an RGB image continuously captured during movement by inputting the third depth map to a convolution long term short memory (LSTM).

Provided are a depth completion method and apparatus using spatial-temporal information. The depth completion apparatus according to the present invention comprises a processor; and a memory connected to the processor, wherein the memory stores program instructions executable by the processor for performing operations comprising receiving an RGB image and a sparse image through a camera and LiDAR, generating a dense first depth map by processing color information of the RGB image through a first branch based on an encoder-decoder, generating a dense second depth map by up-sampling the sparse image through a second branch based on an encoder-decoder, generating a third depth map by fusing the first depth map and the second depth map, and generating a final depth map including a trajectory of a moving object included in an RGB image continuously captured during movement by inputting the third depth map to a convolution long term short memory (LSTM).

In various examples, surface profile estimation and bump detection may be performed based on a three-dimensional (3D) point cloud. The 3D point cloud may be filtered in view of a portion of an environment including drivable free-space, and within a threshold height to factor out other objects or obstacles other than a driving surface and protuberances thereon. The 3D point cloud may be analyzed—e.g., using a sliding window of bounding shapes along a longitudinal or other heading direction—to determine one-dimensional (1D) signal profiles corresponding to heights along the driving surface. The profile itself may be used by a vehicle—e.g., an autonomous or semi-autonomous vehicle—to help in navigating the environment, and/or the profile may be used to detect bumps, humps, and/or other protuberances along the driving surface, in addition to a location, orientation, and geometry thereof.

In various examples, surface profile estimation and bump detection may be performed based on a three-dimensional (3D) point cloud. The 3D point cloud may be filtered in view of a portion of an environment including drivable free-space, and within a threshold height to factor out other objects or obstacles other than a driving surface and protuberances thereon. The 3D point cloud may be analyzed—e.g., using a sliding window of bounding shapes along a longitudinal or other heading direction—to determine one-dimensional (1D) signal profiles corresponding to heights along the driving surface. The profile itself may be used by a vehicle—e.g., an autonomous or semi-autonomous vehicle—to help in navigating the environment, and/or the profile may be used to detect bumps, humps, and/or other protuberances along the driving surface, in addition to a location, orientation, and geometry thereof.

A three-dimensional (3D) object reconstruction neural network system learns to predict a 3D shape representation of an object from a video that includes the object. The 3D reconstruction technique may be used for content creation, such as generation of 3D characters for games, movies, and 3D printing. When 3D characters are generated from video, the content may also include motion of the character, as predicted based on the video. The 3D object construction technique exploits temporal consistency to reconstruct a dynamic 3D representation of the object from an unlabeled video. Specifically, an object in a video has a consistent shape and consistent texture across multiple frames. Texture, base shape, and part correspondence invariance constraints may be applied to fine-tune the neural network system. The reconstruction technique generalizes well—particularly for non-rigid objects.

A three-dimensional (3D) object reconstruction neural network system learns to predict a 3D shape representation of an object from a video that includes the object. The 3D reconstruction technique may be used for content creation, such as generation of 3D characters for games, movies, and 3D printing. When 3D characters are generated from video, the content may also include motion of the character, as predicted based on the video. The 3D object construction technique exploits temporal consistency to reconstruct a dynamic 3D representation of the object from an unlabeled video. Specifically, an object in a video has a consistent shape and consistent texture across multiple frames. Texture, base shape, and part correspondence invariance constraints may be applied to fine-tune the neural network system. The reconstruction technique generalizes well—particularly for non-rigid objects.

Provided is a method, system, and computer program product for avoiding ground blindness in a vehicle. The method includes capturing, with a detection device, a plurality of frames of three-dimensional data over a time period while the vehicle is approaching a landing zone, the plurality of frames of three-dimensional data representing a region associated with the landing zone, generating a rolling point cloud map for the region by combining, with at least one processor during the time period, a subset of the plurality of frames, determining, with the at least one processor, a ground blindness event occurring in the region during the time period, in response to determining the blindness event occurring in the region, excluding at least one frame from the subset of the plurality of frames used to generate the rolling point cloud map for the region, determining, with at least one processor, position data representing a position of the vehicle based on at least one sensor, and generating, with the at least one processor, an output based on the rolling point cloud map and the position data.

A system and method can support image based determination of mobile device location through recognition of surface features for a previously scanned physical environment. The system and method can also support authoring and positioning of augmented reality features in an authoring interface using the same images and positions of surface features that are to be used for subsequent mobile device localization. As a result, mobile devices leveraging those same images and positions of surface features for localization will be more likely to obtain a localization that is consistent with the positioning displayed in the authoring interface. Augmented reality features authored using the same scan of the environment can be reliably displayed to an end user of an augmented reality application in a position consistent with their authoring in a common coordinate system, even though the authoring may have been performed remotely, away from the actual situs of the physical environment.