A system and method for harmonization of vertical projections for displaying of geospatial object data on a mediated reality (MR) device. The method includes: determining an absolute elevation of the MR device relative to a fixed elevation reference; determining an elevation of the MR device relative to physical ground; determining a projection plane using the elevation of the MR device; determining positioning of the projection plane relative to the fixed elevation reference; determining positioning of the geospatial objects with relative vertical positioning data in the mediated reality display relative to the determined projection plane; determining positioning of the geospatial objects with absolute vertical positioning data in the mediated reality display relative to the fixed elevation reference; and outputting the positioned geospatial objects with relative vertical positioning data and the positioned geospatial objects with absolute vertical positioning data to the display of the MR device.

A system for surgical navigation includes a surgical tool, a tracking device configured to track poses of the surgical tool relative to an anatomical object, and a computer. The computer is programmed to perform a constructive sold geometry (CSG) operation using a model of the anatomical object and an accumulation of the tracked poses of the surgical tool relative to the anatomical object and generate a 2D image based on a result of the CSG operation and a viewpoint of a virtual camera by shading fragments of the 2D image based on the result of the CSG operation and the viewpoint of the virtual camera. The CSG operation is independent of the viewpoint of the virtual camera.

A controller for displaying a puncture site of an intra-atrial septum for heart repairs includes a memory and a processor (710). The processor (710) executes instructions (784) to perform a process based on image data of a heart that includes a mitral valve and an intra-atrial septum. The process includes defining a mitral valve annulus plane along a mitral valve annulus of the mitral valve and a normal vector perpendicular to the mitral valve annulus plane. The process also includes defining an offset plane that intersects with the intra-atrial septum and that is parallel to the mitral valve annulus plane. A safe zone for the puncture site is identified and displayed on the intra-atrial septum. The safe zone is between a lower boundary plane (456) and an upper boundary plane (455) that are each parallel to the offset plane by specified distances.

Disclosed herein is a medical system (100, 300) comprising a display (112) and a user interface (114). The execution of machine executable instructions (120) causes a processor (104) to: receive (200) three-dimensional medical image data (122) of an anatomical structure (128, 322); receive (202) a three-dimensional segmentation (124) with one or more reference locations (800); display (204) at least one two-dimensional slice (126) of the three-dimensional medical image data; render (206) a cross section (134) of the three-dimensional segmentation, provide (208) a control element (130) of the user interface that is configured for receiving a one-dimensional position of the at least one reference location along a predetermined one-dimensional path (806); receive (210) the one-dimensional position (137) from the control element; adjust (212) the three-dimensional segmentation (124) using the one-dimensional position; and update (214) the rendering of the cross section of the three-dimensional segmentation.

A method for visualizing a three-dimensional volume for use in a virtual reality environment is performed by uploading two-dimensional images for evaluation, creating planar depictions of the two-dimensional images, and using thresholds to determine if voxels should be drawn. A voxel volume is created from the planar depictions and voxels. A user defines a plane to be used for slicing the voxel volume, and sets values of the plane location and plane normal. The slice plane is placed within the voxel volume and defines a desired remaining portion of the volumetric plane to be displayed. All but the desired remaining portion of the voxel volume is not drawn and the remaining portion is displayed.

Systems and methods are provided for multi-schema analysis of patient specific anatomical features from medical images. The system may receive medical images of a patient and metadata associated with the medical images indicative of a selected pathology, and automatically classify the medical images using a segmentation algorithm. The system may use an anatomical feature identification algorithm to identify one or more patient specific anatomical features within the medical images by exploring an anatomical knowledge dataset. A 3D surface mesh model may be generated representing the one or more classified patient specific anatomical features, such that information may be extracted from the 3D surface mesh model based on the selected pathology. Physiological information associated with the selected pathology for the 3D surface mesh model may be generated based on the extracted information.

Methods, systems, and computer-readable media for generating a cross-section of a 3D model are disclosed. An example method includes determining a cross-section plane intersecting the 3D model, performing ray-tracing by passing each of a plurality of rays through a corresponding pixel of a viewing plane such that each ray intersects the cross-section plane, determining one or more rays that are within a threshold distance of the 3D model at their respective points of intersection with the cross section plane, and highlighting pixels corresponding to the determined rays.

The present disclosure involves object recognition as a method of registration, using a stereoscopic camera on Augmented Reality (AR) glasses or an endoscope as the image capture technology. Exemplary objects include surgical tools, anatomical components or features, such as bone or cartilage, etc. By detecting just a portion of the object in the image data of the surgical scene, the present disclosure may register and track a portion of the patient's anatomy, such as the pelvis, the knee, etc. The present disclosure also optionally displays information on the AR glasses themselves, such as the entire pelvis, the femur, the tibia, etc. The present disclosure may include combinations of the foregoing features, and may eliminate the need for electromagnetic, inertial, or infrared stereoscopic tracking as the tracking technology.

Aspects of the disclosure include methods, apparatuses, and non-transitory computer-readable storage mediums for generating a floor plan from a point cloud model. An apparatus includes processing circuitry that receives an input three-dimensional point cloud corresponding to a three-dimensional space. The processing circuitry determines a plurality of wall planes in the received input three-dimensional point cloud. The processing circuitry generates a plurality of line segments. Each line segment is generated by projecting a respective wall plane of the plurality of wall planes to a floor plane in the three-dimensional space. The processing circuitry represents the plurality of wall planes in the three-dimensional space using the plurality of line segments in a two-dimensional space corresponding to the floor plan. The processing circuitry adjusts the plurality of line segments in the two-dimensional space to improve the floor plan. The processing circuitry generates the floor plan based on the plurality of adjusted line segments.

An example technique for generating slice data from a voxel representation can include obtaining a shape specification of the 3-D object. The example technique for generating slice data from a voxel representation can also include obtaining a material specification of the 3-D object. The example technique for generating slice data from a voxel representation can also include merging the shape specification and the material specification to create a voxel representation of the 3-D object, wherein each voxel in the voxel representation includes a plurality of data types. The example technique for generating slice data from a voxel representation can also include generating slice data from the voxel representation, wherein the slice data provides a higher resolution than that provided by the voxel representation using the plurality of data types.