The disclosed system may include a housing dimensioned to secure various components including at least one physical processor and various sensors. The system may also include a camera mounted to the housing, as well as physical memory with computer-executable instructions that, when executed by the physical processor, cause the physical processor to: acquire images of a surrounding environment using the camera mounted to the housing, identify features of the surrounding environment from the acquired images, generate a map using the features identified from the acquired images, access sensor data generated by the sensors, and determine a current pose of the system in the surrounding environment based on the features in the generated map and the accessed sensor data. Various other methods, apparatuses, and computer-readable media are also disclosed.

A method for creating artificial vision with an implantable visual stimulation device. The method comprises receiving image data comprising, for each of multiple points of an image, a depth value, performing a local background enclosure calculation on the image data to determine salient object information, and generating a visual stimulus to visualise the salient object information using the implantable visual stimulation device. Performing the local background enclosure calculation is based on a subset of the multiple points of the input image, and the subset of the multiple points is defined based on the depth value of the multiple points.

A method for training a depth estimation model with depth hints is disclosed. For each image pair: for a first image, a depth prediction is determined by the depth estimation model and a depth hint is obtained; the second image is projected onto the first image once to generate a synthetic frame based on the depth prediction and again to generate a hinted synthetic frame based on the depth hint; a primary loss is calculated with the synthetic frame; a hinted loss is calculated with the hinted synthetic frame; and an overall loss is calculated for the image pair based on a per-pixel determination of whether the primary loss or the hinted loss is smaller, wherein if the hinted loss is smaller than the primary loss, then the overall loss includes the primary loss and a supervised depth loss between depth prediction and depth hint. The depth estimation model is trained by minimizing the overall losses for the image pairs.

A method comprising obtaining, from a sensor, depth data representing a target object; selecting a model to fit to the depth data; for each data point in the depth data: defining a ray from a location of the sensor to the data point; and determining an error based on a distance from the data point to the model along the ray; when the depth data does not meet a similarity threshold for the model based on the determined errors, selecting a new model and repeating the error determination for the depth data based on the new model; when the depth data meets the similarity threshold for the model, selecting the model as representing the target object; and outputting the selected model representing the target object.

The present invention relates to a medical data processing method of determining the representation of an anatomical body part (2) of a patient (1) in a sequence of medical images, the anatomical body part (2) being subject to a vital movement of the patient (1), the method being constituted to be executed by a computer and comprising the following steps: a) acquiring advance medical image data comprising a time-related advance medical image comprising a representation of the anatomical body part (2) in a specific movement phase; b) acquiring current medical image data describing a sequence of current medical images, wherein the sequence comprises a specific current medical image comprising a representation of the anatomical body part (2) in the specific movement phase, and a tracking current medical image which is different from the specific current medical image and comprises a representation of the anatomical body part (2) in a tracking movement phase which is different from the specific movement phase; c) determining, based on the advance medical image data and the current medical image data, specific image subset data describing a specific image subset of the specific current medical image, the specific image subset comprising the representation of the anatomical body part (2); d) determining, based on the current medical image data and the image subset data, subset tracking data describing a tracked image subset in the tracking current medical image, the tracked image subset comprising the representation of the anatomical body part (2).

A first aspect of the invention provides a method of training a neural network for capturing volumetric video, comprising: generating a 3D model of a scene; using the 3D model to generate a high fidelity depth map; capturing a perceived depth map of the scene, having a field of view that is aligned with a field of view of the high fidelity depth map; and training the neural network based on the high fidelity depth map and the perceived depth map, wherein the high fidelity depth map has a higher fidelity to the scene than the perceived depth map has.

An example control system includes a memory and at least one processor to obtain image data from a given region and perform image analysis on the image data to detect a set of objects in the given region. For each object of the set, the example control system may classify each object as being one of multiple predefined classifications of object permanency, including (i) a fixed classification, (ii) a static and fixed classification, and/or (iii) a dynamic classification. The control system may generate at least a first layer of a occupancy map for the given region that depicts each detected object that is of the static and fixed classification and excluding each detected object that is either of the static and unfixed classification or of the dynamic classification.

A motion detecting section detects a change in relative position relation between a subject and an image capturing section performing a rolling shutter operation. A thinning-out setting section sets a thinning-out amount of a line thinning-out operation of the image capturing section according to the detection result obtained by the motion detecting section. A recognition processing section performs subject recognition in an image obtained by the image capturing section, by using a recognizer corresponding to the thinning-out amount set by the thinning-out setting section. The change in relative position relation is detected based on motion of a moving body on which the image capturing section is mounted, an image capturing scene, an image obtained by the image capturing section, and the like. Line thinning-out is performed during the rolling shutter operation, and the thinning-out amount is set according to the detection result obtained by the motion detecting section.

In various examples, surface profile estimation and bump detection may be performed based on a three-dimensional (3D) point cloud. The 3D point cloud may be filtered in view of a portion of an environment including drivable free-space, and within a threshold height to factor out other objects or obstacles other than a driving surface and protuberances thereon. The 3D point cloud may be analyzed—e.g., using a sliding window of bounding shapes along a longitudinal or other heading direction—to determine one-dimensional (1D) signal profiles corresponding to heights along the driving surface. The profile itself may be used by a vehicle—e.g., an autonomous or semi-autonomous vehicle—to help in navigating the environment, and/or the profile may be used to detect bumps, humps, and/or other protuberances along the driving surface, in addition to a location, orientation, and geometry thereof.

Depth information can be used to assist with image processing functionality, such as image stabilization and blur reduction. In at least some embodiments, depth information obtained from stereo imaging or distance sensing, for example, can be used to determine a foreground object and background object(s) for an image or frame of video. The foreground object then can be located in later frames of video or subsequent images. Small offsets of the foreground object can be determined, and the offset accounted for by adjusting the subsequent frames or images. Such an approach provides image stabilization for at least a foreground object, while providing simplified processing and reduce power consumption. Similarly processes can be used to reduce blur for an identified foreground object in a series of images, where the blur of the identified object is analyzed.