Technologies for 3D virtual environment placement of 3D models based on 2D images are disclosed. At least an outline of a 3D virtual environment may be generated. A 2D image of one or more 2D images may be identified. A first product from the first 2D image may be identified. At least one 3D model of one or more 3D models based, at least, on the first product may be determined. A first location for placement of the first product in the 3D virtual environment may be identified. The at least one 3D model may be added within the 3D virtual environment based, at least, on the first location. The 3D virtual environment may be rendered into a visually interpretable form. A second product may be identified from the first 2D image, forming a first grouping of products. A starting element for the first grouping of products may be determined.

Method, executed by a computer, for identifying a coronary sinus of a patient, comprising: receiving a 3D image of a body region of the patient; extracting 2D axial images of the 3D image taken along respective axial planes, 2D sagittal images of the 3D image taken along respective sagittal planes, and 2D coronal images of the 3D image taken along respective coronal planes; applying an axial neural network to each 2D axial image to generate a respective 2D axial probability map, a sagittal neural network to each 2D sagittal image to generate a respective 2D sagittal probability map, and a coronal neural network to each 2D coronal image to generate a respective 2D coronal probability map; generating, based on the 2D probability maps, a 3D mask of the coronary sinus of the patient.

A representation apparatus for displaying a graphical representation of an augmented reality includes a capture unit, a first display unit, and a processing unit. The first display unit is at least partially transparent. The capture unit is configured to capture a relative positioning of the first display unit relative to a representation area of a second display unit. The processing unit is configured to determine an observation geometry between the first display unit and the representation area of the second display unit based on the relative positioning, receive a dataset, generate the augmented reality based on the dataset, and provide the graphical representation of the augmented reality via virtual mapping of the augmented reality onto the representation area along the observation geometry. The first display unit displays the graphical representation of the augmented reality in at least partial overlaying with the representation area of the second display unit.

For three-dimensional segmentation from two-dimensional intracardiac echocardiography imaging, the three-dimension segmentation is output by a machine-learnt multi-task generator. The machine-learnt multi-task generator is trained from 3D information, such as a sparse ICE volume assembled from the 2D ICE images. The machine-learnt multi-task generator is trained to output both the 3D segmentation and a complete volume. The 3D segmentation may be used to project to 2D as an input with an ICE image to another network trained to output a 2D segmentation for the ICE image. Display of the 3D segmentation and/or 2D segmentation may guide ablation of tissue in the patient.

A method (300) includes receiving a first facial framework (144a) and a first captured image (130a) of a face (20). The first facial framework corresponds to the face at a first frame and includes a first facial mesh (142a) of facial information (140). The method also includes projecting the first captured image onto the first facial framework and determining a facial texture (212) corresponding to the face based on the projected first captured image. The method also includes receiving a second facial framework (144b) at a second frame that includes a second facial mesh (142b) of facial information and updating the facial texture based on the received second facial framework. The method also includes displaying the updated facial texture as a three-dimensional avatar (160). The three-dimensional avatar corresponds to a virtual representation of the face.

A method for generating an intraoperative 3D brain model while a patient is operated. Before an opening in a patient's skull is made, the method includes: providing a preoperative 3D brain model of a patient's brain and converting it to a preoperative 3D brain point cloud; providing a preoperative 3D face model of a patient's face and converting it to a preoperative 3D face point cloud. After the opening in the patient's skull is made, the method includes: matching the intraoperative 3D face point cloud with the preoperative 3D face point cloud to find a face point transformation; transforming the intraoperative 3D brain point cloud based on said face point cloud transformation; comparing the intraoperative 3D brain point cloud with the preoperative 3D brain point cloud to determine a brain shift; and converting the preoperative 3D brain model to generate an intraoperative 3D brain model based on said brain shift.

Three-dimensional points may be projected onto first locations in a first image of an object captured from a first position in three-dimensional space relative to the object and projected onto second locations a virtual camera position located at a second position in three-dimensional space relative to the object. First transformations linking the first and second locations may then be determined. Second transformations transforming first coordinates for the first image to second coordinates for the second image may be determined based on the first transformations. Based on these second transformations and on the first image, a second image of the object from the virtual camera position.

A method for determining an ROI in medical imaging may include receiving first position information related to a body contour of a subject with respect to a support from a flexible device configured with a plurality of position sensors. The flexible device may be configured to conform to the body contour of the subject, and the support may be configured to support the subject. The method may also include generating a 3D model of the subject based on the first position information. The method may further include determining an ROI of the subject based on the 3D model of the subject.

In accordance with some embodiments of the disclosed subject matter, methods, systems, and media for generating images of multiple sides of an object are provided. In some embodiments, a method comprises receiving information indicative of a 3D pose of a first object in a first coordinate space at a first time; receiving a group of images captured using at least one image sensor, each image associated with a field of view within the first coordinate space; mapping at least a portion of a surface of the first object to a 2D area with respect to the image based on the 3D pose of the first object; associating, for images including the surface, a portion of that image with the surface of the first object based on the 2D area; and generating a composite image of the surface using images associated with the surface.

A system for generating three-dimensional models of an exterior physical environment such as the exterior of a building. In some cases, the system may utilize third-party data to supplement data captured at the physical environment when generating the three-dimensional models. The system may also utilize third-party data to assist with aligning models of an exterior of a building with an interior of the building.