A system for determining and visualizing damage to an anatomical joint of a patient. The system is to: obtain a three dimensional image representation of an anatomical joint which is based on a medical image stack; determine damage to an anatomical structure in the anatomical joint by analyzing the medical image stack; mark damage to the anatomical structures in the obtained three dimensional image representation; obtain a 3D model based on the three dimensional image representation; and create a graphical user interface (GUI). The GUI may comprise: functionality to visualize and enable manipulation of the at least one 3D model; functionality to enable removal of the visualization of the anatomical structure from the 3D model; functionality to visualize and enable browsing of the medical image stack; and functionality to visualize the position of the medical image that is currently visualized.

Systems and techniques are provided for modeling three-dimensional (3D) meshes using multi-view image data. An example method can include determining, based on a first image of a target, first 3D mesh parameters for the target corresponding to a first coordinate frame; determining, based on a second image of the target, second 3D mesh parameters for the target corresponding to a second coordinate frame; determining third 3D mesh parameters for the target in a third coordinate frame, the third 3D mesh parameters being based on the first and second 3D mesh parameters and relative rotation and translation parameters of image sensors that captured the first and second images; determining a loss associated with the third 3D mesh parameters, the loss being based on the first and second 3D mesh parameters and the relative rotation and translation parameters; determining 3D mesh parameters based on the loss and third 3D mesh parameters.

Methods and systems are provided for aggregating features in multiple video frames to enhance tissue abnormality detection algorithms, wherein a first detection algorithm identifies an abnormality and aggregates adjacent video frames to create a more complete image for analysis by an artificial intelligence detection algorithm, the aggregation occurring in real time as the medical procedure is being performed.

The described technology is a technique related to a server and method for displaying additional information on a 3D tour. In this case, the method for displaying additional information on a 3D tour performed by a server for providing a 3D tour service linked to a user terminal, the method includes providing a 3D tour comprising 3D information of a particular space to the user terminal, receiving coordinate information for a start point and an end point of an additional-information signpost to be created newly on the 3D tour displayed on the user terminal, creating an additional-information signpost connecting the start point and the end point, and overlappingly displaying the created additional-information signpost on the 3D tour.

Systems and methods for improving endoscopy procedures are described that provide not only a conventional real time image of the view obtained by an endoscope, but in addition, a near real time 3D model and/or a 2D flattened image of an interior surface of an organ, which model and image may be processed using AI software to highlight potential tissue abnormalities for closer examination and/or biopsy during the procedure. A navigation module interacts with other system outputs to further assist the endoscopist with navigational indicia, e.g., landmarks and/or directional arrows, that enhance the endoscopists' spatial orientation, and/or may provide navigational guidance to the endoscopist to assist manipulation of the endoscope.

The embodiments of the present disclosure disclose an augmented reality system that supports customized multi-channel interaction. One embodiment of the augmented reality system comprises: a head-mounted sensor assembly, a computing device, and a display module; the head-mounted sensor assembly is used to capture the user's multi-channel interactive input information and transmit the interactive input information to the computing device; the computing device is used to generate or modify the display content of the augmented reality according to the interactive input information; the display module is used to overlay display the background content with the display content of the augmented reality. The augmented reality system, by arranging the display module to the far end of the head-mounted sensor assembly, can simplify the structure of the head-mounted sensor assembly, and reduce the weight of the head-mounted sensor assembly, providing convenience for installing other sensors. At the same time, the system may incorporate multiple ways of interaction, thereby enriching the system's interaction with the user, and improving the user's experience.

Techniques are described that facilitate co-viewing in an experiential reality environment such as a virtual reality or augmented reality environment. These techniques include prioritizing a main view of experiential reality content over side views of the experiential reality content.

A system, software application and process help the user memorize a digital environment through the process of actions the users have to perform. The embodiments provide generating a digital environment including a first virtual room. A first set of digital objects in pre-defined locations of the first virtual room is generated. The first virtual room and the digital objects in the pre-defined locations are displayed. An acknowledgement from the user that the user has memorized positions of each of the digital objects in the first virtual room is received. The objects are removed. Selection of the digital objects and their placement in the first virtual room is received. The system determines whether every digital object was placed in a correct location and in response, generates a second virtual room including a second set of digital objects for further memory training.

A method and device for measuring dimensions of a feature on or near an object using a video inspection device. A reference surface is determined based on reference surface points on the surface of the object. One or more measurement cursors are placed on measurement pixels of an image of the object. Projected reference surface points associated with the measurement pixels on the reference surface are determined. The dimensions of the feature can be determined using the three-dimensional coordinates of at least one of the projected reference surface points.

A plurality of visualization objects may be provided for representing one or more data sets in a virtual 3D space. The visualization objects may include funnels, containers, name cards, and so forth. The visualization objects can be arranged in a circular carousel that can be rotated around a position of a virtual camera or user in a VR/AR environment. Individual data points in the visualization objects can be rotated, sized, positioned, colored, or otherwise characterized based on attributes of the corresponding data points. Individual data points can also be animated as transitioning between visualization objects in a unified view. Voice commands can be interpreted as part of an interactive environment that can provide views of the visualization objects to multiple devices simultaneously.