A computerized system and method for user interface management provides an augmented reality-based experience that delivers real-time visual cues to a cognitively impaired person, that monitors visual recognition performance, and that provides visual recognition training based on observed visual recognition performance. The system uses image/facial recognition to identify persons/places/objects encountered by a person/dementia sufferer and captured by a camera of a computing device, retrieves information associated with identified person/place/object, and displays retrieved information associated with identified person/place/object to the person/dementia sufferer via the device, e.g., in an augmented reality overlay to an image captured by the camera. The information may identify the identified person's relationship with dementia sufferer and other pertinent details. Accordingly, for example, a person with cognitive impairment can quickly assess who is approaching them, based on facial recognition.

A computerized system and method for user interface management provides an augmented reality-based experience that delivers real-time visual cues to a cognitively impaired person, that monitors visual recognition performance, and that provides visual recognition training based on observed visual recognition performance. The system uses image/facial recognition to identify persons/places/objects encountered by a person/dementia sufferer and captured by a camera of a computing device, retrieves information associated with identified person/place/object, and displays retrieved information associated with identified person/place/object to the person/dementia sufferer via the device, e.g., in an augmented reality overlay to an image captured by the camera. The information may identify the identified person's relationship with dementia sufferer and other pertinent details. Accordingly, for example, a person with cognitive impairment can quickly assess who is approaching them, based on facial recognition.

An exemplary process obtains sensor data corresponding to a physical environment including one or more physical objects. A physical property of the one or more physical objects is determined based on the sensor data. A presentation mode associated with a knowledge domain is determined. An extended reality environment including a view of the physical environment and a visualization selected based on the determined presentation mode is provided. The visualization includes virtual content associated with the knowledge domain. The virtual content is provided based on display characteristics specified by the presentation mode that depend upon the physical property of the one or more objects.

An exemplary process obtains sensor data corresponding to a physical environment including one or more physical objects. A physical property of the one or more physical objects is determined based on the sensor data. A presentation mode associated with a knowledge domain is determined. An extended reality environment including a view of the physical environment and a visualization selected based on the determined presentation mode is provided. The visualization includes virtual content associated with the knowledge domain. The virtual content is provided based on display characteristics specified by the presentation mode that depend upon the physical property of the one or more objects.

A method for performing a procedure on a patient includes acquiring a three-dimensional image of a location of interest on the patient and a two-dimensional image of the location of interest can be acquired. A computer system can relate the three-dimensional image with the two-dimensional image to form a holographic image dataset. The computer system can register the holographic image dataset with the patient. The augmented reality system can render a hologram based on the holographic image dataset from the patient. The hologram can include a projection of the three-dimensional image and a projection of the two-dimensional image. The practitioner can view the hologram with the augmented reality system and perform the procedure on the patient. The practitioner can employ the augmented reality system to visualize a point on the projection of the three-dimensional image and a corresponding point on the projection of the two-dimensional image during the procedure.

A method for performing a procedure on a patient includes acquiring a three-dimensional image of a location of interest on the patient and a two-dimensional image of the location of interest can be acquired. A computer system can relate the three-dimensional image with the two-dimensional image to form a holographic image dataset. The computer system can register the holographic image dataset with the patient. The augmented reality system can render a hologram based on the holographic image dataset from the patient. The hologram can include a projection of the three-dimensional image and a projection of the two-dimensional image. The practitioner can view the hologram with the augmented reality system and perform the procedure on the patient. The practitioner can employ the augmented reality system to visualize a point on the projection of the three-dimensional image and a corresponding point on the projection of the two-dimensional image during the procedure.

A system for accurately positioning augmented reality (AR) content within a coordinate system such as the World Geodetic System (WGS) may include AR content tethered to trackable physical features. As the system is used by mobile computing devices, each mobile device may calculate and compare relative positioning data between the trackable features. The system may connect and group the trackable features hierarchically, as measurements are obtained. As additional measurements are made of the trackable features in a group, the relative position data may be improved, e.g., using statistical methods.

A system for accurately positioning augmented reality (AR) content within a coordinate system such as the World Geodetic System (WGS) may include AR content tethered to trackable physical features. As the system is used by mobile computing devices, each mobile device may calculate and compare relative positioning data between the trackable features. The system may connect and group the trackable features hierarchically, as measurements are obtained. As additional measurements are made of the trackable features in a group, the relative position data may be improved, e.g., using statistical methods.

Systems and methods are disclosed herein for monitoring a location of a client device associated with a transportation service and generating augmented reality images for display on the client device. The systems and methods use sensor data from the client device and a device localization process to monitor the location of the client device by comparing renderings of images captured by the client device to renderings of the vicinity of the pickup location. The systems and methods determine navigation instructions from the user's current location to the pickup location and select one or more augmented reality elements associated with the navigation instructions and/or landmarks along the route to the pickup location. The systems and methods instruct the client device to overlay the selected augmented reality elements on a video feed of the client device.

Systems and methods are disclosed herein for monitoring a location of a client device associated with a transportation service and generating augmented reality images for display on the client device. The systems and methods use sensor data from the client device and a device localization process to monitor the location of the client device by comparing renderings of images captured by the client device to renderings of the vicinity of the pickup location. The systems and methods determine navigation instructions from the user's current location to the pickup location and select one or more augmented reality elements associated with the navigation instructions and/or landmarks along the route to the pickup location. The systems and methods instruct the client device to overlay the selected augmented reality elements on a video feed of the client device.